-- Logs begin at Fri 2022-09-09 22:31:36 UTC, end at Fri 2022-09-09 23:41:10 UTC. -- Sep 09 23:01:16.889272 dv-cinder-850321-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 09 23:01:16.898696 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 9 23:01:16 2022] *** Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: nodename: dv-cinder-850321-6-iscsi Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: machine: x86_64 Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: clock source: unix Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: pcre jit disabled Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: detected number of CPU cores: 4 Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: current working directory: / Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: detected binary path: /usr/bin/uwsgi-core Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: your processes number limit is 47680 Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: your memory page size is 4096 bytes Sep 09 23:01:16.900637 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: detected max file descriptor number: 2048 Sep 09 23:01:16.901380 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: lock engine: pthread robust mutexes Sep 09 23:01:16.901380 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: thunder lock: enabled Sep 09 23:01:16.901380 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 09 23:01:16.901380 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 09 23:01:16.939830 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: Python main interpreter initialized at 0x562597b249c0 Sep 09 23:01:16.939830 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: python threads support enabled Sep 09 23:01:16.939830 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: your server socket listen backlog is limited to 100 connections Sep 09 23:01:16.939830 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: your mercy for graceful operations on workers is 90 seconds Sep 09 23:01:16.940173 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: mapped 403077 bytes (393 KB) for 2 cores Sep 09 23:01:16.940250 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: *** Operational MODE: preforking *** Sep 09 23:01:16.940521 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: *** uWSGI is running in multiple interpreter mode *** Sep 09 23:01:16.940521 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: spawned uWSGI master process (pid: 81758) Sep 09 23:01:16.940719 dv-cinder-850321-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 09 23:01:16.942527 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: spawned uWSGI worker 1 (pid: 81769, cores: 1) Sep 09 23:01:16.942527 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: spawned uWSGI worker 2 (pid: 81770, cores: 1) Sep 09 23:01:16.942527 dv-cinder-850321-6-iscsi devstack@keystone.service[81758]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 09 23:01:21.312969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 23:01:21.312969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: from cryptography.utils import int_from_bytes Sep 09 23:01:21.313753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 23:01:21.313753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: from cryptography.utils import int_from_bytes Sep 09 23:01:21.313917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 23:01:21.313917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: from cryptography.utils import int_from_bytes Sep 09 23:01:21.314498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 09 23:01:21.314498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: from cryptography.utils import int_from_bytes Sep 09 23:01:21.364022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 23:01:21.364193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 23:01:21.364382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] command line args: [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 23:01:21.364648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 23:01:21.364822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ================================================================================ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 23:01:21.365070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] admin_token = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.365311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] config_dir = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.365494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.365673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] config_source = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.365850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] debug = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.366047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.366470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.366614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.366862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.367107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.367334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.367510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_config_append = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.367686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.367865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_dir = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.368052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_file = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.368261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.368503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.368743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.368986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.369161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.369336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.369574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.369814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.370045 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.370297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.370535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.370766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.371024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.371248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.371518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.371999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.372169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] publish_errors = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.372389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.372620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.372794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.372983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 09 23:01:21.373068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.373181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 09 23:01:21.373269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] command line args: [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 09 23:01:21.373460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] standard_threads = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.373646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 09 23:01:21.373724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.373809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ================================================================================ {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 09 23:01:21.373909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] admin_token = **** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_journal = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] config_dir = [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_json = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_stderr = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] config_source = [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] use_syslog = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] debug = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.374970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.375049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.375222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.375309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.375447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.375532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.375699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.375777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.375933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.376015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.376163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] list_limit = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.376251 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.376330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] log_config_append = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.376406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.376507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.376628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.376706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] log_dir = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.376847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] log_file = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.376931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.377082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.377167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.377326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.377414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.377566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.377651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.377802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.377880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.378014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.378106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.378259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.378378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.378503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.378605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.378770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.378857 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.379013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.379098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.379248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.379339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.379481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.379566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.379714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.379807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.379953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.380039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.380196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.380272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.380450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.380527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.380702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.380944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.381032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.381126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] publish_errors = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.381205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.381358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.381438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.381585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.381671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.381765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.381844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.382009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.382088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.382245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.382323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.382506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] standard_threads = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.382593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.382741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.382827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.382983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.383060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.383159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.383242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.383325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] use_journal = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.383450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.383529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] use_json = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.383660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] use_stderr = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.383743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.383823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] use_syslog = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.383905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.384055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 09 23:01:21.384134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.384311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.384390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.384550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.384636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.384774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.384852 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385000 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.385931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.386945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.387110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.387202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.387337 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.387422 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.387570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.387649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.387810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.387920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.388970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.389045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.389204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.389278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.389438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.389513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.389666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.389762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.389916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.390970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.391149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.391229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.391382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.391458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.391624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.391713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.391863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.391950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.392095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.392175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.392327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.392418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.392558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.392638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.392799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.392881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.393949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.394044 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.394729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.394837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.394961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.395086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.395200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.395328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.395449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.395568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.395726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.395816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.395971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.396051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.396204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.396289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.396443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.396532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.396681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.396767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.396930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.397953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.398094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.398173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.398326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.398505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.398587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.398744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.398821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.398893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.399972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.external = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.400991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.401131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.401217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.401405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.401492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.401642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.401774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.401892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.password = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] auth.token = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.402892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.403981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.404061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.404216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.404295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.404455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.404544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.404691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.404777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.404928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405008 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.405961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.406112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.406192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.406370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.406452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.406609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.406689 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.406847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.406930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.407085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.407165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.407316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.407396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.407548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.407629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.407777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.407889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.408947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.409028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.409178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.409261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.409426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.409507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.409661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.409770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.410917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.410917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] auth.external = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.411043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.411103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.417536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.417536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] auth.password = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] auth.token = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.418966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419042 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419345 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] credential.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.419968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.420984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.421995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] federation.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.422938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423164 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.423985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.424969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425113 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.425997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426852 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.426997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.427982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428266 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428473 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.428941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.429992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.430939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.431982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.432137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.432229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.432381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.432464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.432628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.432710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.432861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.432950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.433103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.433185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.433335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.433416 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.433570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.433663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.433802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] role.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.433884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] role.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434263 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.434989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.435079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.435221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.435304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.435459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.435540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.435699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.435789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.435941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.436963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.437124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.437214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.437363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.437443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.437603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.437775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.437865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.438009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.438090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.438251 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.438333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.438526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.438620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.438772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.438925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.439950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.440948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.441658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] resource.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.441750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.441914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.442019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.442145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.442226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.442398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.442483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.442628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.445936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] role.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] role.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446471 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.446988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.447932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.448953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.449976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.450118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.450199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.450405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.450505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.450609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.450691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.450846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.450931 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.451077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.451168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.451244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81769) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 23:01:21.451392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.451632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.451869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.452102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.452340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] token.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.452579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.452862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.453355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.453610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.453860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.454097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.454364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.454610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.454856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.455095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.455334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.455570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.455817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.456094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.456327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.456717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.456989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.457253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.connection = **** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.457501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.457750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.458025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.458237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.458505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.458745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.458988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.459226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.459474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.459713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.459958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.460194 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.460443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.460679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.460926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.461157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.461399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.461634 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 09 23:01:21.461806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81770) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 09 23:01:21.481241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.481463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.481686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.487293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.487528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.487756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.490848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.496874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.518484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.523085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.523168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.523168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.586696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.587649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.636960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.637190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.638638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.638874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.798798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.799922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.800087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.823077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.823307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.823405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.823523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.823608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.823734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.823837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.823941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.824028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.824150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.824233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.824357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.824438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.824639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.836369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.838118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.850033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.850258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.850508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.850733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.850944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.851155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.851366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81769) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.851547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.851780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.852004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.852224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.852441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.852656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.852870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81770) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 09 23:01:21.868093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.868301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 23:01:21.868551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.868640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.868762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 23:01:21.868845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 23:01:21.869071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.869152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.869291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 09 23:01:21.869375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.869565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.869648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.869815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.869904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.870077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.870357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.879372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.879650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.879923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.880020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.880206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.880290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.880503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.880788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.887120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.887746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.889987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.890272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.890582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.890677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.890864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.890955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.891144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.891225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.891402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.891493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.891671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.891761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.891940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.892022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.892203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.892285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.892471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.892554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.892736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.892825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.893024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.893297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.916163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.917241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.917605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.917885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.918743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.919033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.922536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.922817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.923091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.923685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.923973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.924258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.929364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.929648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.930825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.931116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.935248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.935536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.935814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.936092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.936360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.936630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.936757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.936897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.937059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.937162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.937352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.937643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.937927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.938216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.938521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.938797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.952498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.952773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.954526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.954818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.957392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.959228 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.960595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.960888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.962479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.962779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.965047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.966983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.968146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.970172 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.971351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.971651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.971936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.972330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.972674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.972956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.973636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.973941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.974235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.974550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.974837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.975124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.989154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.989437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.989700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.989958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.991511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:21.991802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.992073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.992339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.996736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.998264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.999168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:21.999808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.000079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.000717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.002251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.002558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.002794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.003075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.003355 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.003719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.004104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.005297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.005582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.005873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.006159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.006584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.015074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.015359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.015639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.015920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.016298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.016576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.016863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.017675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.017962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.018246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.018556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.018940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.019761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.020045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.032664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.035686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.035783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.038635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.038741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.040064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.041511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.041593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.041795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.042185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.042986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.044471 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.044757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.045166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.048473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.049918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.052151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.053343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.053605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.053737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.054022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.054299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.056507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.056910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.057197 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.057475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.061992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.062293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.062598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.065265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.065570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.065858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.069505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81769) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.069788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.070069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.070367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.070643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.070913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.071184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.071469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.071742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.072016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.072340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.072598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.072861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.072941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81770) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 09 23:01:22.073133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81769) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.073215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.073417 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.073694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.073968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.074239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.074539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.074832 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.075103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.075372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.075663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.075940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.076233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.076517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81770) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 09 23:01:22.107829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562597b249c0 pid: 81769 (default app) Sep 09 23:01:22.111063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362af677-c8fd-4135-9109-4d9d37d73937 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:22.111231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562597b249c0 pid: 81770 (default app) Sep 09 23:01:22.111347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362af677-c8fd-4135-9109-4d9d37d73937 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:22.111615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362af677-c8fd-4135-9109-4d9d37d73937 None None] PATH_INFO: `/v3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:22.116337 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Fri Sep 9 23:01:22 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:29.829216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9bce-06bd-46be-bbe0-94280c3c0a8d None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:29.829720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9bce-06bd-46be-bbe0-94280c3c0a8d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:29.829806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc9bce-06bd-46be-bbe0-94280c3c0a8d None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:29.834735 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:01:29.839856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:29.840308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:29.840409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:29.911563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: INFO dbcounter [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] Registered counter for database keystone Sep 09 23:01:29.930842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG oslo_db.sqlalchemy.engines [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81769) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 23:01:29.937033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writer thread running {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 23:01:30.120903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG passlib.handlers.bcrypt [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81769) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 23:01:30.121207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG passlib.handlers.bcrypt [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81769) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 23:01:30.179117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:30.180723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:30.309710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 09 23:01:30.780351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc9bce-06bd-46be-bbe0-94280c3c0a8d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:30.857073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: INFO dbcounter [None req-5ccc9bce-06bd-46be-bbe0-94280c3c0a8d None None] Registered counter for database keystone Sep 09 23:01:30.878525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG oslo_db.sqlalchemy.engines [None req-5ccc9bce-06bd-46be-bbe0-94280c3c0a8d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81770) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 09 23:01:30.884884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writer thread running {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 09 23:01:31.003573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc9bce-06bd-46be-bbe0-94280c3c0a8d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:31.008052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb979aa-1037-410b-9649-d37ba07e7167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:31.008673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb979aa-1037-410b-9649-d37ba07e7167 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:31.008954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb979aa-1037-410b-9649-d37ba07e7167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:31.009225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb979aa-1037-410b-9649-d37ba07e7167 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:31.017371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb979aa-1037-410b-9649-d37ba07e7167 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:31.069244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ebb979aa-1037-410b-9649-d37ba07e7167 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 23:01:31.069586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb979aa-1037-410b-9649-d37ba07e7167 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:31.079098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Fri Sep 9 23:01:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:01:31.085970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:31.103256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07af75fa-99b7-4fe2-a47e-5debc5e4e6f9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:31.108043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6675ba5-d5cd-4c7f-8b8d-6447bd76be4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:31.109160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6675ba5-d5cd-4c7f-8b8d-6447bd76be4c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:31.109434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6675ba5-d5cd-4c7f-8b8d-6447bd76be4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:31.109699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6675ba5-d5cd-4c7f-8b8d-6447bd76be4c admin admin] PATH_INFO: `/v3/projects/7393fe084b62454e9d96e9d57737a145` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:31.111335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6675ba5-d5cd-4c7f-8b8d-6447bd76be4c admin admin] RBAC: Authorizing `identity:get_project(project_id=7393fe084b62454e9d96e9d57737a145)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:31.165286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b6675ba5-d5cd-4c7f-8b8d-6447bd76be4c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 09 23:01:31.165627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6675ba5-d5cd-4c7f-8b8d-6447bd76be4c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:31.170265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Fri Sep 9 23:01:31 2022] GET /identity/v3/projects/7393fe084b62454e9d96e9d57737a145 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:33.306168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8984b3-4f84-492c-9c7f-aa18230646db None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:33.306744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8984b3-4f84-492c-9c7f-aa18230646db None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:33.306744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8984b3-4f84-492c-9c7f-aa18230646db None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:33.310866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:33.316381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3246b8-2094-4466-8af2-bcd8e1fe2475 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:33.316766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3246b8-2094-4466-8af2-bcd8e1fe2475 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:33.316943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3246b8-2094-4466-8af2-bcd8e1fe2475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:33.337633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7f3246b8-2094-4466-8af2-bcd8e1fe2475 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:33.339180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7f3246b8-2094-4466-8af2-bcd8e1fe2475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:33.347035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 09 23:01:33.614955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8984b3-4f84-492c-9c7f-aa18230646db None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:33.627213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8984b3-4f84-492c-9c7f-aa18230646db None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:33.632020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd5e3c-321b-4112-bb76-e801e620f5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:33.632614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd5e3c-321b-4112-bb76-e801e620f5ec admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:33.632896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd5e3c-321b-4112-bb76-e801e620f5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:33.633172 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd5e3c-321b-4112-bb76-e801e620f5ec admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:33.640824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd5e3c-321b-4112-bb76-e801e620f5ec admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:33.642611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd5e3c-321b-4112-bb76-e801e620f5ec admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:33.651861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Fri Sep 9 23:01:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:01:33.657214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3246b8-2094-4466-8af2-bcd8e1fe2475 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:33.666277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3246b8-2094-4466-8af2-bcd8e1fe2475 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:33.670625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609336ad-8acc-4135-bf7a-3e8b80325d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:33.671237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609336ad-8acc-4135-bf7a-3e8b80325d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:33.671511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609336ad-8acc-4135-bf7a-3e8b80325d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:33.671778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609336ad-8acc-4135-bf7a-3e8b80325d3d admin admin] PATH_INFO: `/v3/users/e84ee63259eb48d0a32d4ab69da0f378` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:33.673073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609336ad-8acc-4135-bf7a-3e8b80325d3d admin admin] RBAC: Authorizing `identity:get_user(user_id=e84ee63259eb48d0a32d4ab69da0f378)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:33.674935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609336ad-8acc-4135-bf7a-3e8b80325d3d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:33.676443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Fri Sep 9 23:01:33 2022] GET /identity/v3/users/e84ee63259eb48d0a32d4ab69da0f378 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:35.919707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b54b6-93f5-48c1-bbd3-c1a40d9bae51 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:35.920250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b54b6-93f5-48c1-bbd3-c1a40d9bae51 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:35.920250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b54b6-93f5-48c1-bbd3-c1a40d9bae51 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:35.921590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:35.927285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83820630-071b-4010-8a77-13b756e9467d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:35.927683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83820630-071b-4010-8a77-13b756e9467d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:35.927901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83820630-071b-4010-8a77-13b756e9467d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:35.945606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-83820630-071b-4010-8a77-13b756e9467d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:35.946809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-83820630-071b-4010-8a77-13b756e9467d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:35.962319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbddab-f2e4-422f-9119-15cbf0f36ce1 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:35.962631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbddab-f2e4-422f-9119-15cbf0f36ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:35.962908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbddab-f2e4-422f-9119-15cbf0f36ce1 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:35.964460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:01:35.969333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:35.969616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:35.969896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:36.012060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG passlib.handlers.bcrypt [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81770) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 09 23:01:36.012438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG passlib.handlers.bcrypt [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81770) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 09 23:01:36.028011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:01:36.062365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:36.063962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:36.162119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:01:36.300605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83820630-071b-4010-8a77-13b756e9467d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:36.325857 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:36.410485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83820630-071b-4010-8a77-13b756e9467d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:36.433478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:36.447144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b45592-d113-43ee-8c44-1441be4862c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:36.447781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b45592-d113-43ee-8c44-1441be4862c8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:36.448058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b45592-d113-43ee-8c44-1441be4862c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:36.448329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b45592-d113-43ee-8c44-1441be4862c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:36.462580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b45592-d113-43ee-8c44-1441be4862c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-51b45592-d113-43ee-8c44-1441be4862c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:36.471379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:36.472342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:01:36.473151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:01:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:36.478367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b45592-d113-43ee-8c44-1441be4862c8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:36.539504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac351770-4af4-44e4-bf98-99a34fd46ca7 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:36.541757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:36.560057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:36.577421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b45592-d113-43ee-8c44-1441be4862c8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:36.578692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d702e67-4df6-4fe8-83d6-93b9019ef058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:36.579562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d702e67-4df6-4fe8-83d6-93b9019ef058 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:36.579844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d702e67-4df6-4fe8-83d6-93b9019ef058 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:36.580118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d702e67-4df6-4fe8-83d6-93b9019ef058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:36.611205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:36.611808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:36.612088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:36.612362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:36.613028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:36.613333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:36.615633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:36.627834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:36.633444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:36.648198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d702e67-4df6-4fe8-83d6-93b9019ef058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:36.649755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d702e67-4df6-4fe8-83d6-93b9019ef058 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:36.732720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9e678-a386-41e7-acc7-d4be7bdff758 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:36.747089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 8/16] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:01:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 23:01:36.752587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d702e67-4df6-4fe8-83d6-93b9019ef058 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:36.755843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:36.766878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837dad6-2a39-42bf-b87b-78ec5c3deea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:36.767509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837dad6-2a39-42bf-b87b-78ec5c3deea9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:36.767782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837dad6-2a39-42bf-b87b-78ec5c3deea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:36.768052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837dad6-2a39-42bf-b87b-78ec5c3deea9 None admin] PATH_INFO: `/v3/users/e84ee63259eb48d0a32d4ab69da0f378` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:36.769178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837dad6-2a39-42bf-b87b-78ec5c3deea9 None admin] RBAC: Authorizing `identity:get_user(user_id=e84ee63259eb48d0a32d4ab69da0f378)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:36.770695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837dad6-2a39-42bf-b87b-78ec5c3deea9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:36.772262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:36 2022] GET /identity/v3/users/e84ee63259eb48d0a32d4ab69da0f378 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:36.777541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837dad6-2a39-42bf-b87b-78ec5c3deea9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:36.791938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:36.856147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d702e67-4df6-4fe8-83d6-93b9019ef058 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:36.883266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837dad6-2a39-42bf-b87b-78ec5c3deea9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:36.888752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:36.889598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:36.889873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:36.890142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:36.908721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:36.910177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:36.918894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae9b81-b66e-4307-bc42-f1672f206284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:36.919899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae9b81-b66e-4307-bc42-f1672f206284 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:36.920176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae9b81-b66e-4307-bc42-f1672f206284 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:36.920446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae9b81-b66e-4307-bc42-f1672f206284 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:36.921557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae9b81-b66e-4307-bc42-f1672f206284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:36.923020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae9b81-b66e-4307-bc42-f1672f206284 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:36.924562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:01:36.929729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:36.931092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:01:36.932143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:01:36.932143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:01:36.932143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:01:36.932297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae9b81-b66e-4307-bc42-f1672f206284 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:36.932630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:01:36.936334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:36.993580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:37.012690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:37.031696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae9b81-b66e-4307-bc42-f1672f206284 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:37.036874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53703e-f769-4af9-8a31-f7c4a56b2487 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:37.065642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182f720-bad6-497d-a87f-5bc4d9ac6d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:37.066253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182f720-bad6-497d-a87f-5bc4d9ac6d14 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:37.066562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182f720-bad6-497d-a87f-5bc4d9ac6d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:37.066842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182f720-bad6-497d-a87f-5bc4d9ac6d14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:37.067722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182f720-bad6-497d-a87f-5bc4d9ac6d14 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.domain.id=default, user.id=e84ee63259eb48d0a32d4ab69da0f378) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:37.068049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182f720-bad6-497d-a87f-5bc4d9ac6d14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:37.069533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182f720-bad6-497d-a87f-5bc4d9ac6d14 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:37.070978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a036db8-8a7b-46ad-bf72-b2f15f21ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:37.071586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036db8-8a7b-46ad-bf72-b2f15f21ca99 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:37.071864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036db8-8a7b-46ad-bf72-b2f15f21ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:37.072137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a036db8-8a7b-46ad-bf72-b2f15f21ca99 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:37.072780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a036db8-8a7b-46ad-bf72-b2f15f21ca99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:37.073082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a036db8-8a7b-46ad-bf72-b2f15f21ca99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:37.074467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a036db8-8a7b-46ad-bf72-b2f15f21ca99 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:37.076125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Fri Sep 9 23:01:36 2022] GET /identity/v3/role_assignments?user.id=e84ee63259eb48d0a32d4ab69da0f378&scope.domain.id=default&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:37.094934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:39.416291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae98a-c407-407d-8330-a79d143d0243 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:39.416858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae98a-c407-407d-8330-a79d143d0243 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:39.416858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ae98a-c407-407d-8330-a79d143d0243 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:39.418310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:39.426799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf254e-6b39-458b-8d1a-4b617730246b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:39.427373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf254e-6b39-458b-8d1a-4b617730246b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:39.427659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf254e-6b39-458b-8d1a-4b617730246b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:39.458589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-21cf254e-6b39-458b-8d1a-4b617730246b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:39.460030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-21cf254e-6b39-458b-8d1a-4b617730246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:39.550754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 11/23] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:39.596722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915536-0085-4c4d-aa0a-d6e3a52a34b6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:39.597033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915536-0085-4c4d-aa0a-d6e3a52a34b6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:39.597269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915536-0085-4c4d-aa0a-d6e3a52a34b6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:39.598685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 13/24] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:39.605427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ec035-3951-4b67-82f1-7950191982af None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:39.605714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ec035-3951-4b67-82f1-7950191982af None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:39.605994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ec035-3951-4b67-82f1-7950191982af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:39.631737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-229ec035-3951-4b67-82f1-7950191982af None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:39.632968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-229ec035-3951-4b67-82f1-7950191982af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:39.688468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:39.720039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 12/25] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:40.034977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058554-e266-4bd4-ac34-dbfe4e46d991 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.035625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058554-e266-4bd4-ac34-dbfe4e46d991 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.036165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058554-e266-4bd4-ac34-dbfe4e46d991 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.037988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 14/26] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:40.043577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a961e-e1bc-4a25-8b66-0a93ff41fd59 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.044114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a961e-e1bc-4a25-8b66-0a93ff41fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.044596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a961e-e1bc-4a25-8b66-0a93ff41fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.067907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-284a961e-e1bc-4a25-8b66-0a93ff41fd59 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:40.069724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-284a961e-e1bc-4a25-8b66-0a93ff41fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:40.154501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 13/27] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:40.167392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03058554-e266-4bd4-ac34-dbfe4e46d991 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:40.313301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03058554-e266-4bd4-ac34-dbfe4e46d991 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:40.331978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a961e-e1bc-4a25-8b66-0a93ff41fd59 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:40.358125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383154d-18da-4a41-95c4-dd8d40e93b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:40.359068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383154d-18da-4a41-95c4-dd8d40e93b89 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.359068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383154d-18da-4a41-95c4-dd8d40e93b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.359343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383154d-18da-4a41-95c4-dd8d40e93b89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.374672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383154d-18da-4a41-95c4-dd8d40e93b89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c383154d-18da-4a41-95c4-dd8d40e93b89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:40.379997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:40.380966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:40.381711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:01:40.384215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 15/28] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:01:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:40.386123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86f94e-87d1-434f-b695-fb58ef075cc2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.386580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86f94e-87d1-434f-b695-fb58ef075cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.387011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86f94e-87d1-434f-b695-fb58ef075cc2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.388840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 16/29] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:40.394071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86f94e-87d1-434f-b695-fb58ef075cc2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:40.405384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:40.493415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a961e-e1bc-4a25-8b66-0a93ff41fd59 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:40.502107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:40.542805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b759f6b-1331-4a5a-9344-376472d0fc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:40.543440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b759f6b-1331-4a5a-9344-376472d0fc62 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.543720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b759f6b-1331-4a5a-9344-376472d0fc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.543993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b759f6b-1331-4a5a-9344-376472d0fc62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.563865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b759f6b-1331-4a5a-9344-376472d0fc62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:40.565278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b759f6b-1331-4a5a-9344-376472d0fc62 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-6b759f6b-1331-4a5a-9344-376472d0fc62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:01:40.594129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:40.595099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:01:40.595872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:01:40.595872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:01:40.595872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:01:40.595872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 14/30] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:01:40.600546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1922a-e5b7-4f09-b7b5-a05a61f47bf2 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.601094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1922a-e5b7-4f09-b7b5-a05a61f47bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.602489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1922a-e5b7-4f09-b7b5-a05a61f47bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.622485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86f94e-87d1-434f-b695-fb58ef075cc2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:40.626822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6af1922a-e5b7-4f09-b7b5-a05a61f47bf2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:40.628134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6af1922a-e5b7-4f09-b7b5-a05a61f47bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:40.675165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:40.676184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.676708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.677188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.678107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:40.678638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:40.680280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:40.689090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 17/31] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:40.690321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:40.724825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 15/32] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:40.725701 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1922a-e5b7-4f09-b7b5-a05a61f47bf2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:40.772630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:40.815824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0677d-c50c-487a-b29a-532dad258985 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:40.857280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b0078-dd6c-4bae-a1e1-0ec78a1b6be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:40.858314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b0078-dd6c-4bae-a1e1-0ec78a1b6be2 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.858759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b0078-dd6c-4bae-a1e1-0ec78a1b6be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.859184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b0078-dd6c-4bae-a1e1-0ec78a1b6be2 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.860384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b0078-dd6c-4bae-a1e1-0ec78a1b6be2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:40.861916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b0078-dd6c-4bae-a1e1-0ec78a1b6be2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:40.864118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:40.899756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af1922a-e5b7-4f09-b7b5-a05a61f47bf2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:40.936723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:40.937557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:40.937999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:40.938432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:40.939261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:40.939702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:40.941272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:40.953896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 16/33] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:40.955016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:41.057910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b967ce-af90-4512-936c-efde055e8a57 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:41.093180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346830f-6f24-49f5-83b9-903029be10fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:41.094046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346830f-6f24-49f5-83b9-903029be10fd None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.094543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346830f-6f24-49f5-83b9-903029be10fd None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.094968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346830f-6f24-49f5-83b9-903029be10fd None admin] PATH_INFO: `/v3/users/e84ee63259eb48d0a32d4ab69da0f378` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.096601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346830f-6f24-49f5-83b9-903029be10fd None admin] RBAC: Authorizing `identity:get_user(user_id=e84ee63259eb48d0a32d4ab69da0f378)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:41.098174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346830f-6f24-49f5-83b9-903029be10fd None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:41.100667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 17/34] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:40 2022] GET /identity/v3/users/e84ee63259eb48d0a32d4ab69da0f378 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:41.101546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346830f-6f24-49f5-83b9-903029be10fd None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:41.120714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:41.150656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 18/35] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Sep 9 23:01:40 2022] POST /identity/v3/roles => generated 225 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:01:41.159889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b0078-dd6c-4bae-a1e1-0ec78a1b6be2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:41.211121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346830f-6f24-49f5-83b9-903029be10fd None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:41.251453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f432f1-7cd4-4626-845d-7af996f89ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:41.252346 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f432f1-7cd4-4626-845d-7af996f89ec4 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.252771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f432f1-7cd4-4626-845d-7af996f89ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.253176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f432f1-7cd4-4626-845d-7af996f89ec4 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.253978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f432f1-7cd4-4626-845d-7af996f89ec4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:41.255841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f432f1-7cd4-4626-845d-7af996f89ec4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:41.344648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b0078-dd6c-4bae-a1e1-0ec78a1b6be2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:41.374754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:41.383775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6fa13-5f51-4a1d-bfc1-976818e1b86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:41.384433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fa13-5f51-4a1d-bfc1-976818e1b86d None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.384721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fa13-5f51-4a1d-bfc1-976818e1b86d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.384995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6fa13-5f51-4a1d-bfc1-976818e1b86d None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.385676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6fa13-5f51-4a1d-bfc1-976818e1b86d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:41.387250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb6fa13-5f51-4a1d-bfc1-976818e1b86d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:41.472690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 19/36] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Sep 9 23:01:41 2022] POST /identity/v3/roles => generated 219 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:01:41.482398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6fa13-5f51-4a1d-bfc1-976818e1b86d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:41.496796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 18/37] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:01:41 2022] POST /identity/v3/projects => generated 301 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:01:41.498360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d6c06-fd4a-4c4f-8975-6a1bee9d0d1d None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.499270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d6c06-fd4a-4c4f-8975-6a1bee9d0d1d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.499550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d6c06-fd4a-4c4f-8975-6a1bee9d0d1d None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.500941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 19/38] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:41.507222 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c268-78d1-4a9c-abaf-1ad6ec442060 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.507764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c268-78d1-4a9c-abaf-1ad6ec442060 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.508193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449c268-78d1-4a9c-abaf-1ad6ec442060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.533277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0449c268-78d1-4a9c-abaf-1ad6ec442060 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:41.543896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0449c268-78d1-4a9c-abaf-1ad6ec442060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:41.619622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6fa13-5f51-4a1d-bfc1-976818e1b86d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:41.645206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:41.656506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 20/39] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:01:41.658826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fca02-1f05-4508-b635-b3d7cbd7c771 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.659545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fca02-1f05-4508-b635-b3d7cbd7c771 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.660234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fca02-1f05-4508-b635-b3d7cbd7c771 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.662453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 21/40] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:41.669426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee612750-46fc-4837-b733-af45deeeb051 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.670365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee612750-46fc-4837-b733-af45deeeb051 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.671008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee612750-46fc-4837-b733-af45deeeb051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.672777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd88d4-5396-428b-b880-d63b88bfc1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:41.673443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd88d4-5396-428b-b880-d63b88bfc1af None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.673728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd88d4-5396-428b-b880-d63b88bfc1af None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.674050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dd88d4-5396-428b-b880-d63b88bfc1af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.675356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd88d4-5396-428b-b880-d63b88bfc1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:41.678396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dd88d4-5396-428b-b880-d63b88bfc1af None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:41.680204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 20/41] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:41.685988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd88d4-5396-428b-b880-d63b88bfc1af None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:41.700922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ee612750-46fc-4837-b733-af45deeeb051 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:41.702388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ee612750-46fc-4837-b733-af45deeeb051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:41.792283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:41.872583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:41.920476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dd88d4-5396-428b-b880-d63b88bfc1af None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:41.955797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a3728-9fe4-495e-93df-d8c83052c4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:41.956462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a3728-9fe4-495e-93df-d8c83052c4a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:41.956744 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a3728-9fe4-495e-93df-d8c83052c4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:41.957023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a3728-9fe4-495e-93df-d8c83052c4a7 None admin] PATH_INFO: `/v3/domains/default/users/e84ee63259eb48d0a32d4ab69da0f378/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:41.959825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a3728-9fe4-495e-93df-d8c83052c4a7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e84ee63259eb48d0a32d4ab69da0f378, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:41.961268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a3728-9fe4-495e-93df-d8c83052c4a7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:41.972093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee612750-46fc-4837-b733-af45deeeb051 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:42.036883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 21/43] 10.222.0.20 () {58 vars in 1361 bytes} [Fri Sep 9 23:01:41 2022] PUT /identity/v3/domains/default/users/e84ee63259eb48d0a32d4ab69da0f378/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:01:42.073230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a3728-9fe4-495e-93df-d8c83052c4a7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:42.089141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee612750-46fc-4837-b733-af45deeeb051 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:42.121311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:42.132155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa4f62-7dd6-405d-a30b-73f1c5e89a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:42.132676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa4f62-7dd6-405d-a30b-73f1c5e89a95 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:42.132954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa4f62-7dd6-405d-a30b-73f1c5e89a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:42.133228 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa4f62-7dd6-405d-a30b-73f1c5e89a95 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:42.133911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa4f62-7dd6-405d-a30b-73f1c5e89a95 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:42.135526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa4f62-7dd6-405d-a30b-73f1c5e89a95 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:42.194394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 23/44] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Sep 9 23:01:41 2022] POST /identity/v3/roles => generated 223 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:01:42.194634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a3728-9fe4-495e-93df-d8c83052c4a7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:42.211803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:42.237970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a411-8ba5-432d-931f-b194f40a6e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:42.239036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a411-8ba5-432d-931f-b194f40a6e55 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:42.240432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a411-8ba5-432d-931f-b194f40a6e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:42.240710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a411-8ba5-432d-931f-b194f40a6e55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:42.243251 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a411-8ba5-432d-931f-b194f40a6e55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:42.244890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a411-8ba5-432d-931f-b194f40a6e55 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:42.246853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 22/45] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:42.252709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa4f62-7dd6-405d-a30b-73f1c5e89a95 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:42.367475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa4f62-7dd6-405d-a30b-73f1c5e89a95 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:42.404937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae268c-d27c-40e0-9282-615d210017b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:42.405485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae268c-d27c-40e0-9282-615d210017b2 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:42.405773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae268c-d27c-40e0-9282-615d210017b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:42.406034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae268c-d27c-40e0-9282-615d210017b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:42.406737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae268c-d27c-40e0-9282-615d210017b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:42.408184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae268c-d27c-40e0-9282-615d210017b2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:42.495101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 24/46] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:01:42 2022] POST /identity/v3/projects => generated 312 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:01:44.248359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404ab0-71a4-461a-8221-37a7d53fe8ed None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:44.248904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404ab0-71a4-461a-8221-37a7d53fe8ed None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:44.248904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13404ab0-71a4-461a-8221-37a7d53fe8ed None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:44.250308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 23/47] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:44.256338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8f6de-6305-4e47-ba16-b7ff63479028 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:44.256737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8f6de-6305-4e47-ba16-b7ff63479028 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:44.257022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8f6de-6305-4e47-ba16-b7ff63479028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:44.376684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-afa8f6de-6305-4e47-ba16-b7ff63479028 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:44.377896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-afa8f6de-6305-4e47-ba16-b7ff63479028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:44.413280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:44.452764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 25/48] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:44.662426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fb4e4-4d0b-4e10-8fa4-f3031f524412 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:44.662713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fb4e4-4d0b-4e10-8fa4-f3031f524412 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:44.662990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fb4e4-4d0b-4e10-8fa4-f3031f524412 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:44.664346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 24/49] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:44.669665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c7d68-0ce9-4c94-ba99-28db912f79b9 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:44.669949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c7d68-0ce9-4c94-ba99-28db912f79b9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:44.670218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c7d68-0ce9-4c94-ba99-28db912f79b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:44.688022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-613c7d68-0ce9-4c94-ba99-28db912f79b9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:44.689202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-613c7d68-0ce9-4c94-ba99-28db912f79b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:44.721583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fb4e4-4d0b-4e10-8fa4-f3031f524412 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:44.757924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 26/50] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:01:44.823254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fb4e4-4d0b-4e10-8fa4-f3031f524412 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:44.856922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055c8d4-fd32-4553-a652-a3f493af4837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:44.857548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055c8d4-fd32-4553-a652-a3f493af4837 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:44.857915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055c8d4-fd32-4553-a652-a3f493af4837 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:44.858191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055c8d4-fd32-4553-a652-a3f493af4837 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:44.868273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055c8d4-fd32-4553-a652-a3f493af4837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d055c8d4-fd32-4553-a652-a3f493af4837 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:44.873188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:44.875726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:01:44.876452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 25/51] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:01:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:01:44.880754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c7d68-0ce9-4c94-ba99-28db912f79b9 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:44.979893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:44.983071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c7d68-0ce9-4c94-ba99-28db912f79b9 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:45.015466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:45.016068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:45.016348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:45.016622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:45.017281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:45.017583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:45.019028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:45.030117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055c8d4-fd32-4553-a652-a3f493af4837 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:45.032190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 27/52] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:45.038332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:45.076012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:45.134119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055c8d4-fd32-4553-a652-a3f493af4837 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:45.139592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a8017-d43d-40b2-98ef-dd6db98cc982 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:45.168581 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc50cd8-11f1-4827-a11c-7a6bb83eae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:45.169201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc50cd8-11f1-4827-a11c-7a6bb83eae1c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:45.169477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc50cd8-11f1-4827-a11c-7a6bb83eae1c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:45.169749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc50cd8-11f1-4827-a11c-7a6bb83eae1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:45.170863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc50cd8-11f1-4827-a11c-7a6bb83eae1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:45.172513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc50cd8-11f1-4827-a11c-7a6bb83eae1c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:45.172739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acef691-fe78-463b-bf31-1e8ad4d087b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:45.173357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acef691-fe78-463b-bf31-1e8ad4d087b5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:45.173637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acef691-fe78-463b-bf31-1e8ad4d087b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:45.173907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acef691-fe78-463b-bf31-1e8ad4d087b5 None admin] PATH_INFO: `/v3/users/e84ee63259eb48d0a32d4ab69da0f378` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:45.178400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:45.178549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acef691-fe78-463b-bf31-1e8ad4d087b5 None admin] RBAC: Authorizing `identity:get_user(user_id=e84ee63259eb48d0a32d4ab69da0f378)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:45.178549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acef691-fe78-463b-bf31-1e8ad4d087b5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:45.178549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 28/54] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:45 2022] GET /identity/v3/users/e84ee63259eb48d0a32d4ab69da0f378 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:45.180387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc50cd8-11f1-4827-a11c-7a6bb83eae1c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:45.184074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acef691-fe78-463b-bf31-1e8ad4d087b5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:45.215367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:45.283376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc50cd8-11f1-4827-a11c-7a6bb83eae1c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:45.287043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acef691-fe78-463b-bf31-1e8ad4d087b5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:45.293447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:45.316203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae4679-7971-4c6b-bdff-7adcafb5cda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:45.316801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4679-7971-4c6b-bdff-7adcafb5cda2 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:45.317085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4679-7971-4c6b-bdff-7adcafb5cda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:45.317431 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae4679-7971-4c6b-bdff-7adcafb5cda2 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:45.318084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae4679-7971-4c6b-bdff-7adcafb5cda2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:45.319542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae4679-7971-4c6b-bdff-7adcafb5cda2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:45.320770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864359-e58f-46fc-abb1-648bc9e38259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:45.321409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864359-e58f-46fc-abb1-648bc9e38259 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:45.321692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864359-e58f-46fc-abb1-648bc9e38259 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:45.321969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864359-e58f-46fc-abb1-648bc9e38259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:45.323150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe864359-e58f-46fc-abb1-648bc9e38259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:45.324612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe864359-e58f-46fc-abb1-648bc9e38259 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:45.326223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 29/55] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:45.332201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864359-e58f-46fc-abb1-648bc9e38259 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:45.403567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 27/56] 10.222.0.20 () {60 vars in 1140 bytes} [Fri Sep 9 23:01:45 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:01:45.435083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:45.439287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864359-e58f-46fc-abb1-648bc9e38259 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:45.477775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72960d40-5420-4d97-9ed7-7708b2c403fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:45.478419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960d40-5420-4d97-9ed7-7708b2c403fd None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:45.478702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960d40-5420-4d97-9ed7-7708b2c403fd None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:45.478975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72960d40-5420-4d97-9ed7-7708b2c403fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:45.479708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72960d40-5420-4d97-9ed7-7708b2c403fd None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.domain.id=default, user.id=e84ee63259eb48d0a32d4ab69da0f378) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:45.480012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72960d40-5420-4d97-9ed7-7708b2c403fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:45.481457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72960d40-5420-4d97-9ed7-7708b2c403fd None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:45.492426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 30/57] 10.222.0.20 () {56 vars in 1322 bytes} [Fri Sep 9 23:01:45 2022] GET /identity/v3/role_assignments?user.id=e84ee63259eb48d0a32d4ab69da0f378&scope.domain.id=default&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:01:47.566255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b619f1-df8e-45a1-ad0b-e3231dc6b167 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:47.566811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b619f1-df8e-45a1-ad0b-e3231dc6b167 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:47.566912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b619f1-df8e-45a1-ad0b-e3231dc6b167 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:47.568824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 28/58] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:47.574143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571d816-fa60-418f-aa9d-1dd3031ad0f7 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:47.574547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571d816-fa60-418f-aa9d-1dd3031ad0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:47.574717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571d816-fa60-418f-aa9d-1dd3031ad0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:47.592757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d571d816-fa60-418f-aa9d-1dd3031ad0f7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:47.593919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d571d816-fa60-418f-aa9d-1dd3031ad0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:47.657443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:01:47.931774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b619f1-df8e-45a1-ad0b-e3231dc6b167 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:48.031799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b619f1-df8e-45a1-ad0b-e3231dc6b167 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:48.064798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7c3c6-8d3a-4d91-8c1e-13c8cebe241e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:48.065420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c3c6-8d3a-4d91-8c1e-13c8cebe241e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:48.065701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c3c6-8d3a-4d91-8c1e-13c8cebe241e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:48.065975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c3c6-8d3a-4d91-8c1e-13c8cebe241e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:48.067143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7c3c6-8d3a-4d91-8c1e-13c8cebe241e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:48.069185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7c3c6-8d3a-4d91-8c1e-13c8cebe241e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:48.070777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 29/60] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:48.076191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571d816-fa60-418f-aa9d-1dd3031ad0f7 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:48.164812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:48.173206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571d816-fa60-418f-aa9d-1dd3031ad0f7 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:48.206442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dc14a-6d80-4a56-9a69-716279cf44c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:48.207166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dc14a-6d80-4a56-9a69-716279cf44c6 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:48.207651 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dc14a-6d80-4a56-9a69-716279cf44c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:48.208443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dc14a-6d80-4a56-9a69-716279cf44c6 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:48.209552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dc14a-6d80-4a56-9a69-716279cf44c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:48.211036 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dc14a-6d80-4a56-9a69-716279cf44c6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:48.306029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 32/61] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:01:48 2022] POST /identity/v3/users => generated 275 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:01:51.599754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e624fc-daa6-4a5e-b6d3-9c3df0862b4d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:51.600613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e624fc-daa6-4a5e-b6d3-9c3df0862b4d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:51.601022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e624fc-daa6-4a5e-b6d3-9c3df0862b4d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:51.602548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 30/62] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:51.608011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61fd3b-34e2-4c0c-985c-3d4c2fa936b5 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:51.608821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61fd3b-34e2-4c0c-985c-3d4c2fa936b5 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:51.609335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61fd3b-34e2-4c0c-985c-3d4c2fa936b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:51.640944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-9a61fd3b-34e2-4c0c-985c-3d4c2fa936b5 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:51.642470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9a61fd3b-34e2-4c0c-985c-3d4c2fa936b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:51.730474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 33/63] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:51.765712 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891137b-8bab-4df0-b350-f19a53cf2a53 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:51.766390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891137b-8bab-4df0-b350-f19a53cf2a53 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:51.766862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891137b-8bab-4df0-b350-f19a53cf2a53 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:51.768397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 31/64] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:51.773734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91a764-a4c0-4a5d-a056-f909d56cf2b9 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:51.774022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91a764-a4c0-4a5d-a056-f909d56cf2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:51.774295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91a764-a4c0-4a5d-a056-f909d56cf2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:51.798595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2e91a764-a4c0-4a5d-a056-f909d56cf2b9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:51.799805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2e91a764-a4c0-4a5d-a056-f909d56cf2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:51.900934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:51.944218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bf51b-3043-40ab-9475-9abe243fab48 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:51.944218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bf51b-3043-40ab-9475-9abe243fab48 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:51.944218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bf51b-3043-40ab-9475-9abe243fab48 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:51.944218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 32/66] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:51.949423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bff22-8dcb-4af3-893b-01bde3778761 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:51.949718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bff22-8dcb-4af3-893b-01bde3778761 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:51.949994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bff22-8dcb-4af3-893b-01bde3778761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:51.979950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d39bff22-8dcb-4af3-893b-01bde3778761 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:51.981284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d39bff22-8dcb-4af3-893b-01bde3778761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:51.983354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:52.075857 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bf51b-3043-40ab-9475-9abe243fab48 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.077385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 35/67] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:52.124637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:52.224089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bf51b-3043-40ab-9475-9abe243fab48 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:52.261304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67953218-6b31-449d-ad2a-920368b25152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:52.262056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67953218-6b31-449d-ad2a-920368b25152 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.262361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67953218-6b31-449d-ad2a-920368b25152 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.262633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67953218-6b31-449d-ad2a-920368b25152 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.275100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67953218-6b31-449d-ad2a-920368b25152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-67953218-6b31-449d-ad2a-920368b25152 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:52.280444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:52.281370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:01:52.282109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 33/68] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:01:52.288422 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39bff22-8dcb-4af3-893b-01bde3778761 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.344641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67953218-6b31-449d-ad2a-920368b25152 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.402983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39bff22-8dcb-4af3-893b-01bde3778761 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:52.443953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1d1a4-84a5-4c19-af8c-2b8c351e260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:52.444562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d1a4-84a5-4c19-af8c-2b8c351e260f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.444845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d1a4-84a5-4c19-af8c-2b8c351e260f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.445124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1d1a4-84a5-4c19-af8c-2b8c351e260f None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.445782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1d1a4-84a5-4c19-af8c-2b8c351e260f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:52.446084 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1d1a4-84a5-4c19-af8c-2b8c351e260f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:52.447517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1d1a4-84a5-4c19-af8c-2b8c351e260f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:52.448925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:52.453264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:52.456916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 36/69] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:52.458115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1e94-a95a-4baa-8fe3-e99da2dc4c5a None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.458432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1e94-a95a-4baa-8fe3-e99da2dc4c5a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.458712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc1e94-a95a-4baa-8fe3-e99da2dc4c5a None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.458865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67953218-6b31-449d-ad2a-920368b25152 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:52.462383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 37/70] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:52.462383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e92b-521c-4bd8-921f-d93a1eb52ee8 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.462383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e92b-521c-4bd8-921f-d93a1eb52ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.462383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2e92b-521c-4bd8-921f-d93a1eb52ee8 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.462692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 38/71] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:52.463384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2e92b-521c-4bd8-921f-d93a1eb52ee8 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.495301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24040ac4-de2b-405b-924c-27f50d1394a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:52.495940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24040ac4-de2b-405b-924c-27f50d1394a6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.496222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24040ac4-de2b-405b-924c-27f50d1394a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.496496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24040ac4-de2b-405b-924c-27f50d1394a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.507006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24040ac4-de2b-405b-924c-27f50d1394a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-24040ac4-de2b-405b-924c-27f50d1394a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:52.512053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:52.513033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:01:52.513820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 34/72] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:52.514463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24040ac4-de2b-405b-924c-27f50d1394a6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.571306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2e92b-521c-4bd8-921f-d93a1eb52ee8 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:52.605484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124bf21-b552-49ac-9167-25d820115a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:52.606112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124bf21-b552-49ac-9167-25d820115a16 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.606417 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124bf21-b552-49ac-9167-25d820115a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.606695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124bf21-b552-49ac-9167-25d820115a16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.614656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24040ac4-de2b-405b-924c-27f50d1394a6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:52.618719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124bf21-b552-49ac-9167-25d820115a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-9124bf21-b552-49ac-9167-25d820115a16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:52.625086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:52.626056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:01:52.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 39/73] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:52.627739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97512b62-ab6d-451f-9652-d446bb53c490 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.627739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97512b62-ab6d-451f-9652-d446bb53c490 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.627899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97512b62-ab6d-451f-9652-d446bb53c490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.647945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-97512b62-ab6d-451f-9652-d446bb53c490 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:52.649141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-97512b62-ab6d-451f-9652-d446bb53c490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:52.651044 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44cf60-35cf-4b10-9992-342271e27857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:52.651670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cf60-35cf-4b10-9992-342271e27857 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.651951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cf60-35cf-4b10-9992-342271e27857 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.652222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44cf60-35cf-4b10-9992-342271e27857 None admin] PATH_INFO: `/v3/users/e84ee63259eb48d0a32d4ab69da0f378` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.653353 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44cf60-35cf-4b10-9992-342271e27857 None admin] RBAC: Authorizing `identity:get_user(user_id=e84ee63259eb48d0a32d4ab69da0f378)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:52.654824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f44cf60-35cf-4b10-9992-342271e27857 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:52.656381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 35/74] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/users/e84ee63259eb48d0a32d4ab69da0f378 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:52.657476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412c50-1077-4729-be46-5d98ca14341a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.657757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412c50-1077-4729-be46-5d98ca14341a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.658029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412c50-1077-4729-be46-5d98ca14341a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.675480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ed412c50-1077-4729-be46-5d98ca14341a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:52.677595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ed412c50-1077-4729-be46-5d98ca14341a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:52.691148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:52.705835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:52.716878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 40/75] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:52.717687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97512b62-ab6d-451f-9652-d446bb53c490 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.746834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 36/76] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:52.748500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed412c50-1077-4729-be46-5d98ca14341a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.823018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97512b62-ab6d-451f-9652-d446bb53c490 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:52.863109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:52.863820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.864239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.864742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.865504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:52.865933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:52.867512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:52.876082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 41/77] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:52.877099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.897197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed412c50-1077-4729-be46-5d98ca14341a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:52.932415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:52.933089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:52.933372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:52.933701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:52.934378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:52.934694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:52.936062 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:52.948107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:52.948677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 37/78] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:52.955530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:52.968895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.010199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141eaa6-abff-4a68-a938-ddcb4e2226a0 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.045703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb2e5-df95-4ed1-bfdb-a43fc48326b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.046312 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb2e5-df95-4ed1-bfdb-a43fc48326b0 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.046629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb2e5-df95-4ed1-bfdb-a43fc48326b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.046913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb2e5-df95-4ed1-bfdb-a43fc48326b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.068681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737faa7-44cd-4d5e-a1b5-56175130cef3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.102110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b18fb-7e14-428a-8ffc-40a0c9335d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.102704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b18fb-7e14-428a-8ffc-40a0c9335d3c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.102985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b18fb-7e14-428a-8ffc-40a0c9335d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.103288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b18fb-7e14-428a-8ffc-40a0c9335d3c None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.114736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eb2e5-df95-4ed1-bfdb-a43fc48326b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:53.116578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eb2e5-df95-4ed1-bfdb-a43fc48326b0 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:53.116789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b18fb-7e14-428a-8ffc-40a0c9335d3c None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:53.118282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b18fb-7e14-428a-8ffc-40a0c9335d3c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:53.119960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 38/79] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:53.120728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b18fb-7e14-428a-8ffc-40a0c9335d3c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.169781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.187532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.219379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 42/80] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:01:53.220230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb2e5-df95-4ed1-bfdb-a43fc48326b0 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.220858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b18fb-7e14-428a-8ffc-40a0c9335d3c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.256873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764eeb-a8a0-46dc-b52d-e910999113b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.257500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764eeb-a8a0-46dc-b52d-e910999113b8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.257781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764eeb-a8a0-46dc-b52d-e910999113b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.258055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764eeb-a8a0-46dc-b52d-e910999113b8 None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.259231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764eeb-a8a0-46dc-b52d-e910999113b8 None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:53.260689 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764eeb-a8a0-46dc-b52d-e910999113b8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:53.262270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 39/81] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:53 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:53.267238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764eeb-a8a0-46dc-b52d-e910999113b8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.317930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb2e5-df95-4ed1-bfdb-a43fc48326b0 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.351560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e384-9eb8-4139-bfa7-22b61ae133ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.352235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e384-9eb8-4139-bfa7-22b61ae133ac None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.352454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e384-9eb8-4139-bfa7-22b61ae133ac None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.352722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e384-9eb8-4139-bfa7-22b61ae133ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.353815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825e384-9eb8-4139-bfa7-22b61ae133ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:53.355271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825e384-9eb8-4139-bfa7-22b61ae133ac None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:53.356866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 43/82] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:53.357617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e384-9eb8-4139-bfa7-22b61ae133ac None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.367420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764eeb-a8a0-46dc-b52d-e910999113b8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.379443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.401072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8107e-a3a2-4fb7-9b1c-e4dee4570f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.401702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8107e-a3a2-4fb7-9b1c-e4dee4570f34 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.401984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8107e-a3a2-4fb7-9b1c-e4dee4570f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.402259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8107e-a3a2-4fb7-9b1c-e4dee4570f34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.405231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.414064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8107e-a3a2-4fb7-9b1c-e4dee4570f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-85a8107e-a3a2-4fb7-9b1c-e4dee4570f34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:53.419030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:53.419895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:01:53.420827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 40/83] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:53.421931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8107e-a3a2-4fb7-9b1c-e4dee4570f34 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.463634 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e384-9eb8-4139-bfa7-22b61ae133ac None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.496016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60ee65-9910-4a53-b290-6bfb8c6c2703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.496617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60ee65-9910-4a53-b290-6bfb8c6c2703 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.496894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60ee65-9910-4a53-b290-6bfb8c6c2703 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.497165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60ee65-9910-4a53-b290-6bfb8c6c2703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.520503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8107e-a3a2-4fb7-9b1c-e4dee4570f34 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.555679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305379a2-a6fb-4003-9f9c-992f0bd12f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.556308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305379a2-a6fb-4003-9f9c-992f0bd12f09 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.556590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305379a2-a6fb-4003-9f9c-992f0bd12f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.556860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305379a2-a6fb-4003-9f9c-992f0bd12f09 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.558000 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305379a2-a6fb-4003-9f9c-992f0bd12f09 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:53.559483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305379a2-a6fb-4003-9f9c-992f0bd12f09 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:53.561272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 41/84] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:01:53 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:53.561943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305379a2-a6fb-4003-9f9c-992f0bd12f09 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.564842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60ee65-9910-4a53-b290-6bfb8c6c2703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:53.568228 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e60ee65-9910-4a53-b290-6bfb8c6c2703 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:53.588696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.623730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.662931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305379a2-a6fb-4003-9f9c-992f0bd12f09 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.672059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 44/85] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:01:53.672885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60ee65-9910-4a53-b290-6bfb8c6c2703 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.702365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9f2b4-25ec-4c2c-8203-c2647d36c623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.703015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9f2b4-25ec-4c2c-8203-c2647d36c623 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.703309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9f2b4-25ec-4c2c-8203-c2647d36c623 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.703579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9f2b4-25ec-4c2c-8203-c2647d36c623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.769510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9f2b4-25ec-4c2c-8203-c2647d36c623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:53.771239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9f2b4-25ec-4c2c-8203-c2647d36c623 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:53.772141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e60ee65-9910-4a53-b290-6bfb8c6c2703 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:53.795719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.809915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e338e-c465-4d6e-8df8-2719b0fc4046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:53.810552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e338e-c465-4d6e-8df8-2719b0fc4046 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:53.811317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e338e-c465-4d6e-8df8-2719b0fc4046 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:53.811317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10e338e-c465-4d6e-8df8-2719b0fc4046 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:53.811771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e338e-c465-4d6e-8df8-2719b0fc4046 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:53.813188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10e338e-c465-4d6e-8df8-2719b0fc4046 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:53.834651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:53.870661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 42/86] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:01:53.871511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9f2b4-25ec-4c2c-8203-c2647d36c623 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.893697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 45/87] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:01:53 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:01:53.894512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e338e-c465-4d6e-8df8-2719b0fc4046 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:53.984752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9f2b4-25ec-4c2c-8203-c2647d36c623 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:54.015360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10e338e-c465-4d6e-8df8-2719b0fc4046 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:54.022104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:54.022784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:54.023059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:54.023328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:54.024009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:54.024306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:54.025849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:54.034902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 43/88] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:01:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:54.035798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:54.061502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:54.062670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:54.063200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:54.063477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:54.064209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e, user.id=e84ee63259eb48d0a32d4ab69da0f378) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:54.064515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:54.065985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:54.068778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:54.073247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 46/89] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:01:53 2022] GET /identity/v3/role_assignments?user.id=e84ee63259eb48d0a32d4ab69da0f378&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:54.074335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:54.136522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:54.194405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e946d-4696-48e8-96a5-a8d607ffd598 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:54.218842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904e263-b093-4cef-8ad5-6e9e8012b97a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:54.244896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a4f17-addc-4e85-8a26-2338c855a17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:54.245670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a4f17-addc-4e85-8a26-2338c855a17f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:54.245953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a4f17-addc-4e85-8a26-2338c855a17f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:54.246227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a4f17-addc-4e85-8a26-2338c855a17f None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:54.247475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a4f17-addc-4e85-8a26-2338c855a17f None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:54.249214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7a4f17-addc-4e85-8a26-2338c855a17f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:54.253430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 44/90] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:01:54 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:54.261450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a4f17-addc-4e85-8a26-2338c855a17f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:54.277917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed332e-34ca-4e1c-8060-91f7ad4b0364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:54.278586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed332e-34ca-4e1c-8060-91f7ad4b0364 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:54.278863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed332e-34ca-4e1c-8060-91f7ad4b0364 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:54.279132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed332e-34ca-4e1c-8060-91f7ad4b0364 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:54.280842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed332e-34ca-4e1c-8060-91f7ad4b0364 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:54.282808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed332e-34ca-4e1c-8060-91f7ad4b0364 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:54.284486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 47/91] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:01:54 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:54.285281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed332e-34ca-4e1c-8060-91f7ad4b0364 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:54.389867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:54.399278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a4f17-addc-4e85-8a26-2338c855a17f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:54.416186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed332e-34ca-4e1c-8060-91f7ad4b0364 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:54.425356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:54.447858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c41c7b-4a64-4b50-9fa5-0d56da941ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:54.449279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c41c7b-4a64-4b50-9fa5-0d56da941ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:54.450413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c41c7b-4a64-4b50-9fa5-0d56da941ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:54.450805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c41c7b-4a64-4b50-9fa5-0d56da941ba4 None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:54.454375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c41c7b-4a64-4b50-9fa5-0d56da941ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:54.455850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c41c7b-4a64-4b50-9fa5-0d56da941ba4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:54.458427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 45/92] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:54 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:54.462327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c41c7b-4a64-4b50-9fa5-0d56da941ba4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:54.466623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:54.467387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:54.468496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:54.468496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:54.468870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] RBAC: Adding query filter params (role.id=453242bdc829422490366038c62c149a, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e, user.id=8f335b1a4ae84353bed712f1faf9aad1) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:54.469252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:54.470874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:54.479726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 48/93] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:01:54 2022] GET /identity/v3/role_assignments?user.id=8f335b1a4ae84353bed712f1faf9aad1&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=453242bdc829422490366038c62c149a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:54.480717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:54.645096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c41c7b-4a64-4b50-9fa5-0d56da941ba4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:54.651732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5893b-9eb5-430c-9954-833f39ca99f3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:54.685677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:54.686668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:54.687180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:54.687658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:54.688567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] RBAC: Adding query filter params (role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e, user.id=8f335b1a4ae84353bed712f1faf9aad1) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:54.689048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:54.690734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:54.692760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffad4e-e522-47cf-a98a-378eedc05ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:54.693394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffad4e-e522-47cf-a98a-378eedc05ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:54.693790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffad4e-e522-47cf-a98a-378eedc05ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:54.694073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffad4e-e522-47cf-a98a-378eedc05ab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:54.705375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 46/94] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:01:54 2022] GET /identity/v3/role_assignments?user.id=8f335b1a4ae84353bed712f1faf9aad1&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:54.792255 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:54.830034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffad4e-e522-47cf-a98a-378eedc05ab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:54.831602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffad4e-e522-47cf-a98a-378eedc05ab8 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:54.976406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 49/95] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:01:54.981934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:55.019150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:55.096518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54a4b7-7b2d-4944-91dd-9b629642d1e1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:55.135454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df2632-179e-40fe-ae42-239a89a5ef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:55.136298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df2632-179e-40fe-ae42-239a89a5ef47 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:55.136679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df2632-179e-40fe-ae42-239a89a5ef47 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:55.137078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df2632-179e-40fe-ae42-239a89a5ef47 None admin] PATH_INFO: `/v3/projects/e5882dfbd4ba4063933a05744eafaa5f` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:55.138311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df2632-179e-40fe-ae42-239a89a5ef47 None admin] RBAC: Authorizing `identity:get_project(project_id=e5882dfbd4ba4063933a05744eafaa5f)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:55.139867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df2632-179e-40fe-ae42-239a89a5ef47 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:55.141823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 47/96] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:01:54 2022] GET /identity/v3/projects/e5882dfbd4ba4063933a05744eafaa5f => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:55.147614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffad4e-e522-47cf-a98a-378eedc05ab8 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:55.238157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:55.297234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffad4e-e522-47cf-a98a-378eedc05ab8 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:55.343043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441083a7-38d6-4432-8d81-7bbaa241e6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:55.343653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441083a7-38d6-4432-8d81-7bbaa241e6cb None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:55.343934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441083a7-38d6-4432-8d81-7bbaa241e6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:55.344206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441083a7-38d6-4432-8d81-7bbaa241e6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:55.344929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441083a7-38d6-4432-8d81-7bbaa241e6cb None admin] RBAC: Adding query filter params (role.id=453242bdc829422490366038c62c149a, scope.project.id=e5882dfbd4ba4063933a05744eafaa5f, user.id=8f335b1a4ae84353bed712f1faf9aad1) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:55.345235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441083a7-38d6-4432-8d81-7bbaa241e6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:55.346683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441083a7-38d6-4432-8d81-7bbaa241e6cb None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:55.353898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 50/97] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:01:55 2022] GET /identity/v3/role_assignments?user.id=8f335b1a4ae84353bed712f1faf9aad1&scope.project.id=e5882dfbd4ba4063933a05744eafaa5f&role.id=453242bdc829422490366038c62c149a => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:57.013146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc768-274e-457d-ab13-a7425eefddc0 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.013662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc768-274e-457d-ab13-a7425eefddc0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.013662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbc768-274e-457d-ab13-a7425eefddc0 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:57.015056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 48/98] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:57.021623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471de0d-f098-4ce3-bc2f-526af0d9333f None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.022144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471de0d-f098-4ce3-bc2f-526af0d9333f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.022144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471de0d-f098-4ce3-bc2f-526af0d9333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:57.052300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5471de0d-f098-4ce3-bc2f-526af0d9333f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:57.053491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5471de0d-f098-4ce3-bc2f-526af0d9333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:57.156624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 51/99] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:57.424509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4a239-5530-4359-952e-b241eded8f51 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.424509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4a239-5530-4359-952e-b241eded8f51 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.424509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4a239-5530-4359-952e-b241eded8f51 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:57.424509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 49/100] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:57.430954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c364302-6edb-46df-abe2-bc218a333d1e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.431242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c364302-6edb-46df-abe2-bc218a333d1e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.431520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c364302-6edb-46df-abe2-bc218a333d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:57.435646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc412-e4a4-4c37-81d1-fc01b58cda24 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.436176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc412-e4a4-4c37-81d1-fc01b58cda24 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.436641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc412-e4a4-4c37-81d1-fc01b58cda24 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:57.438191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 50/101] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:01:57.443592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899266b4-d163-46d5-b074-eb62311013eb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.444121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899266b4-d163-46d5-b074-eb62311013eb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.444630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899266b4-d163-46d5-b074-eb62311013eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:57.464558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-899266b4-d163-46d5-b074-eb62311013eb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:57.465480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7c364302-6edb-46df-abe2-bc218a333d1e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:57.466297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-899266b4-d163-46d5-b074-eb62311013eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:57.466688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7c364302-6edb-46df-abe2-bc218a333d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:57.561087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:57.576917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 52/102] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:57.579372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 51/103] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:01:57.667258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c364302-6edb-46df-abe2-bc218a333d1e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:57.822113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c364302-6edb-46df-abe2-bc218a333d1e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:57.873952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd2146-2580-4099-b35a-af22cd3df048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:57.874616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2146-2580-4099-b35a-af22cd3df048 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.874899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2146-2580-4099-b35a-af22cd3df048 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.875162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd2146-2580-4099-b35a-af22cd3df048 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:57.891188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd2146-2580-4099-b35a-af22cd3df048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-60fd2146-2580-4099-b35a-af22cd3df048 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:57.896653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:57.897508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:01:57.898241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 53/104] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:57.906940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899266b4-d163-46d5-b074-eb62311013eb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:57.956552 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:57.982144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a75a-2df4-4e6a-9468-4bd5404044a4 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.982778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a75a-2df4-4e6a-9468-4bd5404044a4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.983845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a75a-2df4-4e6a-9468-4bd5404044a4 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:57.985442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 54/105] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:01:57.990872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3ce67-5ef8-4922-9163-159c17acc69c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:57.991501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3ce67-5ef8-4922-9163-159c17acc69c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:57.991977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3ce67-5ef8-4922-9163-159c17acc69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.040615 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c8a3ce67-5ef8-4922-9163-159c17acc69c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:58.045934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c8a3ce67-5ef8-4922-9163-159c17acc69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:58.108148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899266b4-d163-46d5-b074-eb62311013eb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.115526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:58.134034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 55/106] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:01:58.137703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3ce67-5ef8-4922-9163-159c17acc69c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.159383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.160321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.160822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.161289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.163393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:58.163928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:58.165734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:58.174904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 52/107] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:01:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:58.176086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.251104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3ce67-5ef8-4922-9163-159c17acc69c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.286933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:58.289825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bae030-fcc4-4952-afd5-7e627d5f5e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.290647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bae030-fcc4-4952-afd5-7e627d5f5e62 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.290938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bae030-fcc4-4952-afd5-7e627d5f5e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.291214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bae030-fcc4-4952-afd5-7e627d5f5e62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.304300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bae030-fcc4-4952-afd5-7e627d5f5e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:58.306163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95166e53-c154-4741-853c-4c4caf0aa99e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-62bae030-fcc4-4952-afd5-7e627d5f5e62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:58.309402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:58.310398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:01:58.311542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 56/108] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:58.312165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10cb18-e993-4b17-af23-2530cc9bdfd7 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.312165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10cb18-e993-4b17-af23-2530cc9bdfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.312513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10cb18-e993-4b17-af23-2530cc9bdfd7 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.319683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 57/109] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:58.320495 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10cb18-e993-4b17-af23-2530cc9bdfd7 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.359269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5354d9-a363-437d-aa15-f6161a50956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.363383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5354d9-a363-437d-aa15-f6161a50956d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.364538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5354d9-a363-437d-aa15-f6161a50956d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.364980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5354d9-a363-437d-aa15-f6161a50956d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.377492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5354d9-a363-437d-aa15-f6161a50956d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-df5354d9-a363-437d-aa15-f6161a50956d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:58.384418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:58.385299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:01:58.386052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:01:58.388589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 53/110] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:58.389706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:58.390934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5354d9-a363-437d-aa15-f6161a50956d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.454951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10cb18-e993-4b17-af23-2530cc9bdfd7 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.491624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9364244a-6d88-4a75-9663-5645468c6198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.492259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364244a-6d88-4a75-9663-5645468c6198 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.492536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364244a-6d88-4a75-9663-5645468c6198 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.492937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364244a-6d88-4a75-9663-5645468c6198 None admin] PATH_INFO: `/v3/users/7d8804b93f6b45e6bd4437b6c4272c23` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.506454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5354d9-a363-437d-aa15-f6161a50956d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.511109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364244a-6d88-4a75-9663-5645468c6198 None admin] RBAC: Authorizing `identity:get_user(user_id=7d8804b93f6b45e6bd4437b6c4272c23)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:58.512567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9364244a-6d88-4a75-9663-5645468c6198 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:58.518838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 58/111] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/users/7d8804b93f6b45e6bd4437b6c4272c23 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:58.518838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750fc6d-19a9-46f7-a089-8890e211b0ff None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.518838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750fc6d-19a9-46f7-a089-8890e211b0ff None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.518838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750fc6d-19a9-46f7-a089-8890e211b0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.535467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d750fc6d-19a9-46f7-a089-8890e211b0ff None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:58.537275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d750fc6d-19a9-46f7-a089-8890e211b0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:58.556894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.557517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.557801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.558078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.558788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:58.559093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:58.560480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:58.570300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 54/112] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:58.571397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.582740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:58.626368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 59/113] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:58.626368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750fc6d-19a9-46f7-a089-8890e211b0ff None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.669021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:58.707055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7973c-050d-4390-91bf-a7bf4267f390 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.746991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750fc6d-19a9-46f7-a089-8890e211b0ff None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.760205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.760844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.761195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.761787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.763077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:58.763522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:58.765499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:58.778637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 55/114] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:58.779673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.790023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37c006-4b30-4e97-a4dc-9e048678a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.790815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37c006-4b30-4e97-a4dc-9e048678a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.791124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37c006-4b30-4e97-a4dc-9e048678a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.791401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37c006-4b30-4e97-a4dc-9e048678a5a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.792532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37c006-4b30-4e97-a4dc-9e048678a5a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:58.794113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37c006-4b30-4e97-a4dc-9e048678a5a1 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:58.796206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 60/115] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:58.797047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37c006-4b30-4e97-a4dc-9e048678a5a1 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.856529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:58.888166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff1294-ea24-44fd-98b3-9d61ae4e93d5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.907553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37c006-4b30-4e97-a4dc-9e048678a5a1 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:58.909570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:58.924054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f8424-e732-40dc-b180-960c01729375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.924671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8424-e732-40dc-b180-960c01729375 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.924956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8424-e732-40dc-b180-960c01729375 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.925228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8424-e732-40dc-b180-960c01729375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.945773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561e8c-9b36-49e7-b319-601aab0ca149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:58.946433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561e8c-9b36-49e7-b319-601aab0ca149 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:58.946719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561e8c-9b36-49e7-b319-601aab0ca149 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:58.946995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561e8c-9b36-49e7-b319-601aab0ca149 None admin] PATH_INFO: `/v3/users/e84ee63259eb48d0a32d4ab69da0f378` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:58.948119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a561e8c-9b36-49e7-b319-601aab0ca149 None admin] RBAC: Authorizing `identity:get_user(user_id=e84ee63259eb48d0a32d4ab69da0f378)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:58.949601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a561e8c-9b36-49e7-b319-601aab0ca149 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:58.951241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 61/116] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/users/e84ee63259eb48d0a32d4ab69da0f378 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:58.952005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561e8c-9b36-49e7-b319-601aab0ca149 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:58.995827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f8424-e732-40dc-b180-960c01729375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:58.997281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f8424-e732-40dc-b180-960c01729375 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:59.057584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561e8c-9b36-49e7-b319-601aab0ca149 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.074631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:59.091435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212dda7-4507-4b7c-9cd7-a0a195b9648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.092059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212dda7-4507-4b7c-9cd7-a0a195b9648f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.092338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212dda7-4507-4b7c-9cd7-a0a195b9648f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.092610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212dda7-4507-4b7c-9cd7-a0a195b9648f None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.093715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212dda7-4507-4b7c-9cd7-a0a195b9648f None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:59.095179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212dda7-4507-4b7c-9cd7-a0a195b9648f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:59.096752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 62/117] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:59.098823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212dda7-4507-4b7c-9cd7-a0a195b9648f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.106957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 56/118] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:01:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:01:59.108524 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8146-8583-4893-a834-828ae1923804 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.109060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8146-8583-4893-a834-828ae1923804 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.109060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f8146-8583-4893-a834-828ae1923804 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.110333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 57/119] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:01:59.111483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f8146-8583-4893-a834-828ae1923804 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.141805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:59.201372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212dda7-4507-4b7c-9cd7-a0a195b9648f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.215574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f8146-8583-4893-a834-828ae1923804 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.236128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd32b3-8c44-4231-af2e-2c007d1af87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.236737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd32b3-8c44-4231-af2e-2c007d1af87f None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.237022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd32b3-8c44-4231-af2e-2c007d1af87f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.237298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd32b3-8c44-4231-af2e-2c007d1af87f None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.237937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd32b3-8c44-4231-af2e-2c007d1af87f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:59.240103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd32b3-8c44-4231-af2e-2c007d1af87f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:59.252239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ddd46-85f4-4ee6-a9aa-0478283c8864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.252865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ddd46-85f4-4ee6-a9aa-0478283c8864 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.253147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ddd46-85f4-4ee6-a9aa-0478283c8864 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.253422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ddd46-85f4-4ee6-a9aa-0478283c8864 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.263511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ddd46-85f4-4ee6-a9aa-0478283c8864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2a2ddd46-85f4-4ee6-a9aa-0478283c8864 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:01:59.268302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:01:59.273218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 58/120] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:01:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:01:59.273958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ddd46-85f4-4ee6-a9aa-0478283c8864 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.302140 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:59.352815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 63/121] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:01:59 2022] POST /identity/v3/projects => generated 302 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:01:59.353686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd32b3-8c44-4231-af2e-2c007d1af87f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.377131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ddd46-85f4-4ee6-a9aa-0478283c8864 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.418116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0e858-e815-4d65-8c68-3a0291754cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.419227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0e858-e815-4d65-8c68-3a0291754cba None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.419649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0e858-e815-4d65-8c68-3a0291754cba None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.420190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0e858-e815-4d65-8c68-3a0291754cba None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.421705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0e858-e815-4d65-8c68-3a0291754cba None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:59.423555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0e858-e815-4d65-8c68-3a0291754cba None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:59.430673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 59/122] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:01:59 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:59.430673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0e858-e815-4d65-8c68-3a0291754cba None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.472408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:59.485267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd32b3-8c44-4231-af2e-2c007d1af87f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.526029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7642ed-77c5-4e8c-9c2c-752b30babcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.526732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7642ed-77c5-4e8c-9c2c-752b30babcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.527210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7642ed-77c5-4e8c-9c2c-752b30babcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.527814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7642ed-77c5-4e8c-9c2c-752b30babcc1 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.529263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7642ed-77c5-4e8c-9c2c-752b30babcc1 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:59.530726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7642ed-77c5-4e8c-9c2c-752b30babcc1 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:59.538395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 64/123] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:01:59 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:59.538395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09e21-c4aa-493c-a4a5-ba7a0a20df6e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.538395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09e21-c4aa-493c-a4a5-ba7a0a20df6e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.538395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09e21-c4aa-493c-a4a5-ba7a0a20df6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.538727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0e858-e815-4d65-8c68-3a0291754cba None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.540464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:59.554930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-20c09e21-c4aa-493c-a4a5-ba7a0a20df6e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:01:59.556148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-20c09e21-c4aa-493c-a4a5-ba7a0a20df6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:01:59.578646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cde95c-433b-465d-8d8e-7e8bc8efe84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.579933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cde95c-433b-465d-8d8e-7e8bc8efe84a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.580405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cde95c-433b-465d-8d8e-7e8bc8efe84a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.580915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cde95c-433b-465d-8d8e-7e8bc8efe84a None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.582224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cde95c-433b-465d-8d8e-7e8bc8efe84a None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:59.584581 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cde95c-433b-465d-8d8e-7e8bc8efe84a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:59.586266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 60/124] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:01:59 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:59.587155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cde95c-433b-465d-8d8e-7e8bc8efe84a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.638460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 65/125] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:01:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:01:59.643235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c09e21-c4aa-493c-a4a5-ba7a0a20df6e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.701118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cde95c-433b-465d-8d8e-7e8bc8efe84a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.740302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.741292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.741763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.742289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.743270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:01:59.743783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:59.745486 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:59.750048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:59.760574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 61/126] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:01:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:01:59.761760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.763567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c09e21-c4aa-493c-a4a5-ba7a0a20df6e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.808253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:01:59.814859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ece6a5-0a90-4b01-8fae-043f3916f02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.816501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece6a5-0a90-4b01-8fae-043f3916f02a None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.816786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece6a5-0a90-4b01-8fae-043f3916f02a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.817596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ece6a5-0a90-4b01-8fae-043f3916f02a None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/users/e84ee63259eb48d0a32d4ab69da0f378/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.819943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ece6a5-0a90-4b01-8fae-043f3916f02a None admin] RBAC: Authorizing `identity:create_grant(project_id=e8be5bc6beaa4f6f926f28d045911f2e, user_id=e84ee63259eb48d0a32d4ab69da0f378, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:59.821462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ece6a5-0a90-4b01-8fae-043f3916f02a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:01:59.886645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091cf3c-9957-42e7-8cfb-3e93265333e2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:01:59.902176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 66/127] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:01:59 2022] PUT /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/users/e84ee63259eb48d0a32d4ab69da0f378/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:01:59.903732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ece6a5-0a90-4b01-8fae-043f3916f02a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:01:59.944558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844294-093c-49b6-b688-faabacbc2037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:01:59.945262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844294-093c-49b6-b688-faabacbc2037 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:01:59.945821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844294-093c-49b6-b688-faabacbc2037 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:01:59.945821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36844294-093c-49b6-b688-faabacbc2037 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/users/8f335b1a4ae84353bed712f1faf9aad1/roles/54e2dc31e78b4516b2296086e9eb12c4` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:01:59.948079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36844294-093c-49b6-b688-faabacbc2037 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8be5bc6beaa4f6f926f28d045911f2e, user_id=8f335b1a4ae84353bed712f1faf9aad1, role_id=54e2dc31e78b4516b2296086e9eb12c4)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:01:59.949673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36844294-093c-49b6-b688-faabacbc2037 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:00.032873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ece6a5-0a90-4b01-8fae-043f3916f02a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:00.045648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 62/128] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:01:59 2022] PUT /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/users/8f335b1a4ae84353bed712f1faf9aad1/roles/54e2dc31e78b4516b2296086e9eb12c4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:00.054102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844294-093c-49b6-b688-faabacbc2037 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:00.105576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:00.106166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:00.106530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:00.106806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:00.107730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] RBAC: Adding query filter params (role.id=be22e1353d404cea9407c96a9def8c8c, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e, user.id=7d8804b93f6b45e6bd4437b6c4272c23) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:00.107983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:00.109751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:00.117516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 67/129] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:01:59 2022] GET /identity/v3/role_assignments?user.id=7d8804b93f6b45e6bd4437b6c4272c23&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=be22e1353d404cea9407c96a9def8c8c => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:00.118468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:00.136579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:00.184527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:00.216180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36844294-093c-49b6-b688-faabacbc2037 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:00.302969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb7b33-efc9-4997-b896-9558eaf1e0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:00.303632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb7b33-efc9-4997-b896-9558eaf1e0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:00.303913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb7b33-efc9-4997-b896-9558eaf1e0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:00.304186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb7b33-efc9-4997-b896-9558eaf1e0f4 None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:00.305443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb7b33-efc9-4997-b896-9558eaf1e0f4 None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:00.307106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb7b33-efc9-4997-b896-9558eaf1e0f4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:00.308687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa01fb5-fc92-4440-9a43-df16e2c2e9de None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:00.308809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 63/130] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:00 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:00.321532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb7b33-efc9-4997-b896-9558eaf1e0f4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:00.352792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b0b3-f332-493d-b8ef-1b0bbe8191c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:00.353438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b0b3-f332-493d-b8ef-1b0bbe8191c3 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:00.353718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b0b3-f332-493d-b8ef-1b0bbe8191c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:00.353990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3b0b3-f332-493d-b8ef-1b0bbe8191c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:00.367443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3b0b3-f332-493d-b8ef-1b0bbe8191c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-e3e3b0b3-f332-493d-b8ef-1b0bbe8191c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:00.372411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:00.373283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:00.374030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 68/131] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:00.381606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b0b3-f332-493d-b8ef-1b0bbe8191c3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:00.453654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb7b33-efc9-4997-b896-9558eaf1e0f4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:00.462184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:00.493243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae9c17-2a36-4c3a-8b76-e3a84211f562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:00.493880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9c17-2a36-4c3a-8b76-e3a84211f562 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:00.494158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9c17-2a36-4c3a-8b76-e3a84211f562 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:00.494464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9c17-2a36-4c3a-8b76-e3a84211f562 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:00.494850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3b0b3-f332-493d-b8ef-1b0bbe8191c3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:00.495596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae9c17-2a36-4c3a-8b76-e3a84211f562 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:00.497115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae9c17-2a36-4c3a-8b76-e3a84211f562 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:00.498999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 64/132] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:00 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:00.506427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae9c17-2a36-4c3a-8b76-e3a84211f562 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:00.561491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:00.571912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:00.573486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:00.573486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:00.573486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:00.573913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:00.577056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:00.580587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:00.600177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 69/133] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:00.606814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:00.715647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae9c17-2a36-4c3a-8b76-e3a84211f562 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:00.787749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3933b1-1df4-4c29-b104-245365ede3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:00.788432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3933b1-1df4-4c29-b104-245365ede3bb None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:00.788711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3933b1-1df4-4c29-b104-245365ede3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:00.788994 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3933b1-1df4-4c29-b104-245365ede3bb None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/users/8f335b1a4ae84353bed712f1faf9aad1/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:00.795711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3933b1-1df4-4c29-b104-245365ede3bb None admin] RBAC: Authorizing `identity:create_grant(project_id=e8be5bc6beaa4f6f926f28d045911f2e, user_id=8f335b1a4ae84353bed712f1faf9aad1, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:00.797182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3933b1-1df4-4c29-b104-245365ede3bb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:00.805364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e56d44-717a-4fdc-a09d-b8308da100c5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:00.852547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94231ba-8b5d-4e46-91f8-8762d236f071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:00.856873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94231ba-8b5d-4e46-91f8-8762d236f071 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:00.858041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94231ba-8b5d-4e46-91f8-8762d236f071 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:00.860463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 65/134] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:02:00 2022] PUT /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/users/8f335b1a4ae84353bed712f1faf9aad1/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:00.860578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94231ba-8b5d-4e46-91f8-8762d236f071 None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:00.866774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94231ba-8b5d-4e46-91f8-8762d236f071 None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:00.868332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94231ba-8b5d-4e46-91f8-8762d236f071 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:00.870207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 70/135] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:00 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:00.875754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3933b1-1df4-4c29-b104-245365ede3bb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:00.974980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:00.993955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3933b1-1df4-4c29-b104-245365ede3bb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:01.041118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afd203-33dd-4920-aaf0-31bd36667ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:01.041780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd203-33dd-4920-aaf0-31bd36667ede None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:01.042058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd203-33dd-4920-aaf0-31bd36667ede None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:01.042334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd203-33dd-4920-aaf0-31bd36667ede None admin] PATH_INFO: `/v3/projects/e5882dfbd4ba4063933a05744eafaa5f` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:01.043570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afd203-33dd-4920-aaf0-31bd36667ede None admin] RBAC: Authorizing `identity:get_project(project_id=e5882dfbd4ba4063933a05744eafaa5f)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:01.045035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afd203-33dd-4920-aaf0-31bd36667ede None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:01.047182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 66/136] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:00 2022] GET /identity/v3/projects/e5882dfbd4ba4063933a05744eafaa5f => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:01.053656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94231ba-8b5d-4e46-91f8-8762d236f071 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:01.056763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:01.177907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94231ba-8b5d-4e46-91f8-8762d236f071 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:01.216672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5303687-a76f-4287-9378-a1f59950b395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:01.217337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5303687-a76f-4287-9378-a1f59950b395 None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:01.217620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5303687-a76f-4287-9378-a1f59950b395 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:01.217896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5303687-a76f-4287-9378-a1f59950b395 None admin] PATH_INFO: `/v3/projects/e5882dfbd4ba4063933a05744eafaa5f/users/8f335b1a4ae84353bed712f1faf9aad1/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:01.219922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5303687-a76f-4287-9378-a1f59950b395 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5882dfbd4ba4063933a05744eafaa5f, user_id=8f335b1a4ae84353bed712f1faf9aad1, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:01.221371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5303687-a76f-4287-9378-a1f59950b395 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:01.296888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 71/137] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:02:01 2022] PUT /identity/v3/projects/e5882dfbd4ba4063933a05744eafaa5f/users/8f335b1a4ae84353bed712f1faf9aad1/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:02.333845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eb035-665a-43fe-9d29-a565e3f56744 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:02.334385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eb035-665a-43fe-9d29-a565e3f56744 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:02.334385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1eb035-665a-43fe-9d29-a565e3f56744 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:02.335745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 67/138] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:02.341014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3a0da-fbcf-4348-acb6-65df33ffc24b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:02.341431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3a0da-fbcf-4348-acb6-65df33ffc24b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:02.341650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3a0da-fbcf-4348-acb6-65df33ffc24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:02.367368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-89f3a0da-fbcf-4348-acb6-65df33ffc24b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:02.368601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-89f3a0da-fbcf-4348-acb6-65df33ffc24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:02.445915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:02.460630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 72/139] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:02.883265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c5261-66e2-41ae-81d6-8f8180098c4b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:02.883633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c5261-66e2-41ae-81d6-8f8180098c4b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:02.883828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c5261-66e2-41ae-81d6-8f8180098c4b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:02.885183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 68/140] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:02.890379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dba6f-3822-49e7-88c2-20ecd3336c01 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:02.890678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dba6f-3822-49e7-88c2-20ecd3336c01 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:02.890954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dba6f-3822-49e7-88c2-20ecd3336c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:02.915381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-522dba6f-3822-49e7-88c2-20ecd3336c01 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:02.916666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-522dba6f-3822-49e7-88c2-20ecd3336c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:03.008742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 73/141] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:03.050016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c5261-66e2-41ae-81d6-8f8180098c4b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:03.191418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c5261-66e2-41ae-81d6-8f8180098c4b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:03.202990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ba49f-e117-41d0-acdc-17e2ecf95cca None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:03.203290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ba49f-e117-41d0-acdc-17e2ecf95cca None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:03.203544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ba49f-e117-41d0-acdc-17e2ecf95cca None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:03.205327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 74/142] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:02:03.211895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b6331-b789-4ddb-8a2e-8a2a07bf9e30 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:03.212224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b6331-b789-4ddb-8a2e-8a2a07bf9e30 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:03.212505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b6331-b789-4ddb-8a2e-8a2a07bf9e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:03.241747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-1e3b6331-b789-4ddb-8a2e-8a2a07bf9e30 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:03.242974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1e3b6331-b789-4ddb-8a2e-8a2a07bf9e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:03.257450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017656-b671-4465-acc2-8d7f33cc1dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:03.258090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017656-b671-4465-acc2-8d7f33cc1dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:03.258406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017656-b671-4465-acc2-8d7f33cc1dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:03.258759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd017656-b671-4465-acc2-8d7f33cc1dd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:03.259884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd017656-b671-4465-acc2-8d7f33cc1dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:03.261861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd017656-b671-4465-acc2-8d7f33cc1dd4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:03.266389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 69/143] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:03.269927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017656-b671-4465-acc2-8d7f33cc1dd4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:03.301239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:03.402707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd017656-b671-4465-acc2-8d7f33cc1dd4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:03.459179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 75/144] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:03.460282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f485a2-f67a-45a4-a5c3-0e11595a2b03 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:03.460561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f485a2-f67a-45a4-a5c3-0e11595a2b03 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:03.460831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f485a2-f67a-45a4-a5c3-0e11595a2b03 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:03.462029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de0b3-b243-4a15-b2d1-adf15a210ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:03.466137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 76/145] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:03.467099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de0b3-b243-4a15-b2d1-adf15a210ede None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:03.467623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de0b3-b243-4a15-b2d1-adf15a210ede None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:03.468105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984de0b3-b243-4a15-b2d1-adf15a210ede None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:03.468974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de0b3-b243-4a15-b2d1-adf15a210ede None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:03.470651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984de0b3-b243-4a15-b2d1-adf15a210ede None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:03.479742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093633f2-83a4-45c2-b64d-15ac170fc91d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:03.480022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093633f2-83a4-45c2-b64d-15ac170fc91d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:03.480297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093633f2-83a4-45c2-b64d-15ac170fc91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:03.508644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-093633f2-83a4-45c2-b64d-15ac170fc91d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:03.510205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-093633f2-83a4-45c2-b64d-15ac170fc91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:03.560297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:03.574712 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 70/146] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:03 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:02:03.575900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de0b3-b243-4a15-b2d1-adf15a210ede None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:03.615335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 77/147] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:03.695455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:03.723763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984de0b3-b243-4a15-b2d1-adf15a210ede None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:03.768844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6493aae-f941-497d-9886-81091908f4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:03.769901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493aae-f941-497d-9886-81091908f4dc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:03.771093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493aae-f941-497d-9886-81091908f4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:03.771622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493aae-f941-497d-9886-81091908f4dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:03.786827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6493aae-f941-497d-9886-81091908f4dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f6493aae-f941-497d-9886-81091908f4dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:03.792587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:03.793454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:03.794185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:03.799874 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 71/148] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:03.807027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093633f2-83a4-45c2-b64d-15ac170fc91d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:03.921583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6493aae-f941-497d-9886-81091908f4dc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:03.933946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093633f2-83a4-45c2-b64d-15ac170fc91d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:03.999944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.000588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.000868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.001200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.001871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:04.002228 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:04.003640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:04.016103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 78/149] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:04.017021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.076284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:04.097223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6493aae-f941-497d-9886-81091908f4dc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.142592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b2a3d-a362-47c5-a9b7-3c89a43718a2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.142776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debecbe1-49e5-4022-8d5f-db548d470d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.143419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debecbe1-49e5-4022-8d5f-db548d470d5b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.143696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debecbe1-49e5-4022-8d5f-db548d470d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.143968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debecbe1-49e5-4022-8d5f-db548d470d5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.148051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:04.158864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debecbe1-49e5-4022-8d5f-db548d470d5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-debecbe1-49e5-4022-8d5f-db548d470d5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:04.165488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:04.166625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:04.167363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:04.167922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 72/150] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:04.169419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debecbe1-49e5-4022-8d5f-db548d470d5b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.198001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12472b-bd50-4085-8d3b-0ad58e07c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.199074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12472b-bd50-4085-8d3b-0ad58e07c87c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.199549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12472b-bd50-4085-8d3b-0ad58e07c87c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.200865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12472b-bd50-4085-8d3b-0ad58e07c87c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.221042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12472b-bd50-4085-8d3b-0ad58e07c87c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-4a12472b-bd50-4085-8d3b-0ad58e07c87c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:04.228542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:04.229407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:04.230142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:04.232554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 79/151] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:02:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:04.233500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12472b-bd50-4085-8d3b-0ad58e07c87c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.330860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debecbe1-49e5-4022-8d5f-db548d470d5b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.372699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12472b-bd50-4085-8d3b-0ad58e07c87c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.376152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:04.383259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d09f21-c671-4da0-9282-98d73e5de13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.387458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09f21-c671-4da0-9282-98d73e5de13f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.387458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09f21-c671-4da0-9282-98d73e5de13f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.387458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d09f21-c671-4da0-9282-98d73e5de13f None admin] PATH_INFO: `/v3/users/7d8804b93f6b45e6bd4437b6c4272c23` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.387458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d09f21-c671-4da0-9282-98d73e5de13f None admin] RBAC: Authorizing `identity:get_user(user_id=7d8804b93f6b45e6bd4437b6c4272c23)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:04.388932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d09f21-c671-4da0-9282-98d73e5de13f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:04.391344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 73/152] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:04 2022] GET /identity/v3/users/7d8804b93f6b45e6bd4437b6c4272c23 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:04.392577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d09f21-c671-4da0-9282-98d73e5de13f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.426482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.426482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.426482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.426482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.426838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:04.426838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:04.428938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:04.438728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 80/153] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:04.439556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.475753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:04.528584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d09f21-c671-4da0-9282-98d73e5de13f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.572799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d72df4-089a-4ab8-afd2-39db8145c9dc None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.613625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd3854-21a1-4d0e-805b-57956fdebea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.615769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd3854-21a1-4d0e-805b-57956fdebea9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.616165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd3854-21a1-4d0e-805b-57956fdebea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.616519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd3854-21a1-4d0e-805b-57956fdebea9 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.617805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd3854-21a1-4d0e-805b-57956fdebea9 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:04.619663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd3854-21a1-4d0e-805b-57956fdebea9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:04.621745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 81/154] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:04 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:04.623258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.623902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.624184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.624456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.625141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:04.625439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:04.626996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:04.627960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd3854-21a1-4d0e-805b-57956fdebea9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.652636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 74/155] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:04.653496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.678772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:04.789866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd3854-21a1-4d0e-805b-57956fdebea9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.814660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c4a34-3871-4af5-9765-1f29b501ab24 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.838635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986552c-a0fa-4d13-bcdb-54c3fa1fc5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.839276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986552c-a0fa-4d13-bcdb-54c3fa1fc5ad None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.839553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986552c-a0fa-4d13-bcdb-54c3fa1fc5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.839822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986552c-a0fa-4d13-bcdb-54c3fa1fc5ad None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.841800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986552c-a0fa-4d13-bcdb-54c3fa1fc5ad None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:04.843285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d986552c-a0fa-4d13-bcdb-54c3fa1fc5ad None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:04.845185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 82/156] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:04 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:04.852117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:04.852265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986552c-a0fa-4d13-bcdb-54c3fa1fc5ad None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.861491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de5fd3-ceea-4be1-ae57-3335fde291c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:04.862172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de5fd3-ceea-4be1-ae57-3335fde291c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.862564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de5fd3-ceea-4be1-ae57-3335fde291c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.868970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de5fd3-ceea-4be1-ae57-3335fde291c3 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/users/7d8804b93f6b45e6bd4437b6c4272c23/roles/be22e1353d404cea9407c96a9def8c8c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.868970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de5fd3-ceea-4be1-ae57-3335fde291c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8be5bc6beaa4f6f926f28d045911f2e, user_id=7d8804b93f6b45e6bd4437b6c4272c23, role_id=be22e1353d404cea9407c96a9def8c8c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:04.868970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de5fd3-ceea-4be1-ae57-3335fde291c3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:04.952561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 75/157] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:02:04 2022] PUT /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/users/7d8804b93f6b45e6bd4437b6c4272c23/roles/be22e1353d404cea9407c96a9def8c8c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:04.953839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf982e9-59b5-4345-af45-65fa21750ae6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.954417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf982e9-59b5-4345-af45-65fa21750ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.954690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf982e9-59b5-4345-af45-65fa21750ae6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.959083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 76/158] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:04.959083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28161c71-8f82-449e-9325-bae42a26f0d1 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:04.959083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28161c71-8f82-449e-9325-bae42a26f0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:04.959083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28161c71-8f82-449e-9325-bae42a26f0d1 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:04.959083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 77/159] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:04.959658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28161c71-8f82-449e-9325-bae42a26f0d1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:04.974047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986552c-a0fa-4d13-bcdb-54c3fa1fc5ad None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:04.977162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:05.018959 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ee6c-4146-4b64-ba23-6df4ed8982f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:05.019603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ee6c-4146-4b64-ba23-6df4ed8982f1 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:05.019927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ee6c-4146-4b64-ba23-6df4ed8982f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:05.020292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ee6c-4146-4b64-ba23-6df4ed8982f1 None admin] PATH_INFO: `/v3/users/e84ee63259eb48d0a32d4ab69da0f378` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:05.022287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2ee6c-4146-4b64-ba23-6df4ed8982f1 None admin] RBAC: Authorizing `identity:get_user(user_id=e84ee63259eb48d0a32d4ab69da0f378)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:05.023795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2ee6c-4146-4b64-ba23-6df4ed8982f1 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:05.025855 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 83/160] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:04 2022] GET /identity/v3/users/e84ee63259eb48d0a32d4ab69da0f378 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:05.030447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c63029-1f31-4467-ba65-2fa7a47c6589 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:05.030447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c63029-1f31-4467-ba65-2fa7a47c6589 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:05.030447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c63029-1f31-4467-ba65-2fa7a47c6589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:05.055006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-95c63029-1f31-4467-ba65-2fa7a47c6589 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:05.056223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-95c63029-1f31-4467-ba65-2fa7a47c6589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:05.088240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28161c71-8f82-449e-9325-bae42a26f0d1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:05.125940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390851-48f2-40d7-897a-d9e105ede395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:05.126941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390851-48f2-40d7-897a-d9e105ede395 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:05.127434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390851-48f2-40d7-897a-d9e105ede395 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:05.127919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390851-48f2-40d7-897a-d9e105ede395 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:05.160667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 84/161] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:05.162989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8da0a5-461b-4489-9d92-69082aa3555f None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:05.163273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8da0a5-461b-4489-9d92-69082aa3555f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:05.163540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8da0a5-461b-4489-9d92-69082aa3555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:05.186668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2d8da0a5-461b-4489-9d92-69082aa3555f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:05.188243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2d8da0a5-461b-4489-9d92-69082aa3555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:05.197020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:05.217032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51390851-48f2-40d7-897a-d9e105ede395 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:05.218965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51390851-48f2-40d7-897a-d9e105ede395 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:05.304875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 85/162] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:05.306201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8da0a5-461b-4489-9d92-69082aa3555f None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:05.377177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:05.485377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 78/163] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:05.492384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390851-48f2-40d7-897a-d9e105ede395 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:05.530756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8da0a5-461b-4489-9d92-69082aa3555f None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:05.593813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a912-0257-4760-b80e-0ac3205ed432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:05.594446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a912-0257-4760-b80e-0ac3205ed432 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:05.594715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a912-0257-4760-b80e-0ac3205ed432 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:05.594979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a912-0257-4760-b80e-0ac3205ed432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:05.709418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:05.729172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390851-48f2-40d7-897a-d9e105ede395 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:05.738443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775a912-0257-4760-b80e-0ac3205ed432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:05.739985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775a912-0257-4760-b80e-0ac3205ed432 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:05.786145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d23a8-2c75-4285-9ba0-bf1b3548aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:05.786841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d23a8-2c75-4285-9ba0-bf1b3548aa19 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:05.787121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d23a8-2c75-4285-9ba0-bf1b3548aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:05.787393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d23a8-2c75-4285-9ba0-bf1b3548aa19 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:05.789149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d23a8-2c75-4285-9ba0-bf1b3548aa19 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:05.790625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d23a8-2c75-4285-9ba0-bf1b3548aa19 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:05.794455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 79/164] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:05 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:05.794592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:05.794791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d23a8-2c75-4285-9ba0-bf1b3548aa19 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:05.918284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 86/165] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:05.919163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a912-0257-4760-b80e-0ac3205ed432 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:05.964727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d23a8-2c75-4285-9ba0-bf1b3548aa19 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:06.008562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776636d-bb88-4361-9f4e-1dce5661d9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:06.009672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776636d-bb88-4361-9f4e-1dce5661d9ee None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:06.009965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776636d-bb88-4361-9f4e-1dce5661d9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:06.010235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776636d-bb88-4361-9f4e-1dce5661d9ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:06.024218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c776636d-bb88-4361-9f4e-1dce5661d9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c776636d-bb88-4361-9f4e-1dce5661d9ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:06.031695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:06.032559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:06.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 80/166] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:06.034499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776636d-bb88-4361-9f4e-1dce5661d9ee None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:06.038737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a912-0257-4760-b80e-0ac3205ed432 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:06.054282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:06.076174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:06.087480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:06.088446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:06.088986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:06.089916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:06.090938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] RBAC: Adding query filter params (role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e, user.id=8f335b1a4ae84353bed712f1faf9aad1) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:06.091402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:06.093957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:06.104152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 87/167] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:05 2022] GET /identity/v3/role_assignments?user.id=8f335b1a4ae84353bed712f1faf9aad1&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:06.105827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:06.159190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c776636d-bb88-4361-9f4e-1dce5661d9ee None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:06.198302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a60542-73a0-4c33-b067-681270175645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:06.198965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a60542-73a0-4c33-b067-681270175645 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:06.199245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a60542-73a0-4c33-b067-681270175645 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:06.199518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a60542-73a0-4c33-b067-681270175645 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:06.212003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a60542-73a0-4c33-b067-681270175645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f2a60542-73a0-4c33-b067-681270175645 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:06.217222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:06.218073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:06.218872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 81/168] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:06.219509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a60542-73a0-4c33-b067-681270175645 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:06.257159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2bf1a4-c7a7-4e94-a8b8-71c87d3a479f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:06.311396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:06.322450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961736-2a20-4fca-9c8e-d7581aeb1659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:06.323115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961736-2a20-4fca-9c8e-d7581aeb1659 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:06.323637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961736-2a20-4fca-9c8e-d7581aeb1659 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:06.323916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961736-2a20-4fca-9c8e-d7581aeb1659 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:06.326997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99961736-2a20-4fca-9c8e-d7581aeb1659 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:06.330135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99961736-2a20-4fca-9c8e-d7581aeb1659 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:06.332050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 88/169] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:06 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:06.333960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961736-2a20-4fca-9c8e-d7581aeb1659 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:06.376604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a60542-73a0-4c33-b067-681270175645 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:06.453188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:06.473256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:06.474514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:06.474796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:06.475066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:06.475736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:06.476034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:06.477440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:06.505935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 82/170] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:06.506859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:06.518169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961736-2a20-4fca-9c8e-d7581aeb1659 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:06.570270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:06.570915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:06.571250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:06.571765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:06.572439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:06.572738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:06.574706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:06.589398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 89/171] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:06.590250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:06.746082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835320c2-6abd-47cd-8287-c65dec9874f9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:06.777839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f41fc9-7b0b-4fb3-897a-cd39d9c754f5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:06.794100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:06.819488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9305-b971-405a-bfb7-35f986b47050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:06.820499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9305-b971-405a-bfb7-35f986b47050 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:06.820968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9305-b971-405a-bfb7-35f986b47050 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:06.821425 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9305-b971-405a-bfb7-35f986b47050 None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:06.822836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab9305-b971-405a-bfb7-35f986b47050 None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:06.824624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab9305-b971-405a-bfb7-35f986b47050 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:06.826441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 90/172] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:06 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:06.827453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9305-b971-405a-bfb7-35f986b47050 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:06.851397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:06.852074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:06.852349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:06.852668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:06.853409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e, user.id=e84ee63259eb48d0a32d4ab69da0f378) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:06.853713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:06.855239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:06.864425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:06.872568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 83/173] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:06 2022] GET /identity/v3/role_assignments?user.id=e84ee63259eb48d0a32d4ab69da0f378&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 608 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:06.873391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:07.049778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c381c-50a2-4e15-9949-b41f59b111cd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:07.095255 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9305-b971-405a-bfb7-35f986b47050 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:07.132137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e887a-7569-424b-a152-ae623421d2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:07.132777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e887a-7569-424b-a152-ae623421d2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:07.133053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e887a-7569-424b-a152-ae623421d2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:07.133319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e887a-7569-424b-a152-ae623421d2c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:07.181839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56075b-7255-4caf-8a2c-026b946afc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:07.182881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56075b-7255-4caf-8a2c-026b946afc3b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:07.183352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56075b-7255-4caf-8a2c-026b946afc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:07.183811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56075b-7255-4caf-8a2c-026b946afc3b None admin] PATH_INFO: `/v3/users/8f335b1a4ae84353bed712f1faf9aad1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:07.185192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56075b-7255-4caf-8a2c-026b946afc3b None admin] RBAC: Authorizing `identity:get_user(user_id=8f335b1a4ae84353bed712f1faf9aad1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:07.186877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56075b-7255-4caf-8a2c-026b946afc3b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:07.188725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 91/174] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:06 2022] GET /identity/v3/users/8f335b1a4ae84353bed712f1faf9aad1 => generated 275 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:07.195680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56075b-7255-4caf-8a2c-026b946afc3b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:07.249972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:07.251676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8e887a-7569-424b-a152-ae623421d2c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:07.253156 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8e887a-7569-424b-a152-ae623421d2c9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:07.351962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:07.410482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 84/175] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:07.416911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e887a-7569-424b-a152-ae623421d2c9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:07.461424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56075b-7255-4caf-8a2c-026b946afc3b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:07.534774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b2c40-887c-45fd-912e-d1257a2dd161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:07.535386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2c40-887c-45fd-912e-d1257a2dd161 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:07.535683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2c40-887c-45fd-912e-d1257a2dd161 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:07.535999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b2c40-887c-45fd-912e-d1257a2dd161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:07.600346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:07.604910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e887a-7569-424b-a152-ae623421d2c9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:07.669226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b2c40-887c-45fd-912e-d1257a2dd161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:07.670706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b2c40-887c-45fd-912e-d1257a2dd161 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:07.674977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66679d85-4224-4dbb-a82c-80c9b644ae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:07.675612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66679d85-4224-4dbb-a82c-80c9b644ae05 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:07.675883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66679d85-4224-4dbb-a82c-80c9b644ae05 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:07.676149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66679d85-4224-4dbb-a82c-80c9b644ae05 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:07.679420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66679d85-4224-4dbb-a82c-80c9b644ae05 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:07.680876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66679d85-4224-4dbb-a82c-80c9b644ae05 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:07.682565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 85/176] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:07 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:07.689369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66679d85-4224-4dbb-a82c-80c9b644ae05 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:07.803432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:07.833255 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 92/177] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 23:02:07.838931 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b2c40-887c-45fd-912e-d1257a2dd161 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:07.984403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66679d85-4224-4dbb-a82c-80c9b644ae05 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:08.006058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b2c40-887c-45fd-912e-d1257a2dd161 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:08.038943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad22f8-685b-4265-b98b-3f3c587c8830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:08.039968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad22f8-685b-4265-b98b-3f3c587c8830 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.040441 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad22f8-685b-4265-b98b-3f3c587c8830 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.040905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad22f8-685b-4265-b98b-3f3c587c8830 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.041851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad22f8-685b-4265-b98b-3f3c587c8830 None admin] RBAC: Adding query filter params (role.id=453242bdc829422490366038c62c149a, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e, user.id=8f335b1a4ae84353bed712f1faf9aad1) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:08.042369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad22f8-685b-4265-b98b-3f3c587c8830 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:08.044407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad22f8-685b-4265-b98b-3f3c587c8830 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:08.052891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 86/178] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:07 2022] GET /identity/v3/role_assignments?user.id=8f335b1a4ae84353bed712f1faf9aad1&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=453242bdc829422490366038c62c149a => generated 608 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:08.054979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b50a-0cb5-490c-ac99-0d93fd123118 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.055915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b50a-0cb5-490c-ac99-0d93fd123118 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.056410 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7b50a-0cb5-490c-ac99-0d93fd123118 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.058198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 87/179] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:08.067918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ebeab-58cd-4be0-8f49-5bb95a0550f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:08.069027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ebeab-58cd-4be0-8f49-5bb95a0550f6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.069027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ebeab-58cd-4be0-8f49-5bb95a0550f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.069213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ebeab-58cd-4be0-8f49-5bb95a0550f6 None admin] PATH_INFO: `/v3/projects/e5882dfbd4ba4063933a05744eafaa5f` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.070269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ebeab-58cd-4be0-8f49-5bb95a0550f6 None admin] RBAC: Authorizing `identity:get_project(project_id=e5882dfbd4ba4063933a05744eafaa5f)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:08.070660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c467b32-760c-4598-a43d-5be4f988e62d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.071135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c467b32-760c-4598-a43d-5be4f988e62d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.071630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c467b32-760c-4598-a43d-5be4f988e62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.071730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ebeab-58cd-4be0-8f49-5bb95a0550f6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:08.073447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 93/180] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:07 2022] GET /identity/v3/projects/e5882dfbd4ba4063933a05744eafaa5f => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:08.079459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ebeab-58cd-4be0-8f49-5bb95a0550f6 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:08.106745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7c467b32-760c-4598-a43d-5be4f988e62d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:08.107963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7c467b32-760c-4598-a43d-5be4f988e62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:08.127397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:08.172473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:08.209889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ebeab-58cd-4be0-8f49-5bb95a0550f6 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:08.334433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 88/181] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:08.334433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419d5b7-9199-4503-8de9-521bc2a408a1 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.334433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419d5b7-9199-4503-8de9-521bc2a408a1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.334433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419d5b7-9199-4503-8de9-521bc2a408a1 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.334433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 89/182] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:08.334433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffcbb-15d5-49af-91c8-bc34c1c0bd2a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.334433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffcbb-15d5-49af-91c8-bc34c1c0bd2a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.335105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ffcbb-15d5-49af-91c8-bc34c1c0bd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.335199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51044-0c41-457a-a58f-7d92bf8237eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:08.335199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51044-0c41-457a-a58f-7d92bf8237eb None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.335199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51044-0c41-457a-a58f-7d92bf8237eb None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.335199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51044-0c41-457a-a58f-7d92bf8237eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.335594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51044-0c41-457a-a58f-7d92bf8237eb None admin] RBAC: Adding query filter params (role.id=453242bdc829422490366038c62c149a, scope.project.id=e5882dfbd4ba4063933a05744eafaa5f, user.id=8f335b1a4ae84353bed712f1faf9aad1) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:08.335594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51044-0c41-457a-a58f-7d92bf8237eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:08.335594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51044-0c41-457a-a58f-7d92bf8237eb None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:08.335594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 94/183] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:08 2022] GET /identity/v3/role_assignments?user.id=8f335b1a4ae84353bed712f1faf9aad1&scope.project.id=e5882dfbd4ba4063933a05744eafaa5f&role.id=453242bdc829422490366038c62c149a => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:08.336048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-108ffcbb-15d5-49af-91c8-bc34c1c0bd2a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:08.336048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-108ffcbb-15d5-49af-91c8-bc34c1c0bd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:08.436175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 90/184] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:08.654218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b1ff-d391-4643-81af-8e98f029af4c None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.654477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b1ff-d391-4643-81af-8e98f029af4c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.655034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b1ff-d391-4643-81af-8e98f029af4c None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.656574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 95/185] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:08.663158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f927-adc1-443f-953f-17a28d2b65ef None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.664015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f927-adc1-443f-953f-17a28d2b65ef None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.664532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54f927-adc1-443f-953f-17a28d2b65ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.686603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3e54f927-adc1-443f-953f-17a28d2b65ef None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:08.688096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3e54f927-adc1-443f-953f-17a28d2b65ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:08.719207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b1ff-d391-4643-81af-8e98f029af4c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:08.780933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 91/186] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:08.831842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f927-adc1-443f-953f-17a28d2b65ef None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:08.840988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:08.851409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b1ff-d391-4643-81af-8e98f029af4c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:08.857811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:08.890910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb966cc2-3fa6-4c57-870c-6978db8dcb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:08.891546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966cc2-3fa6-4c57-870c-6978db8dcb33 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.891825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966cc2-3fa6-4c57-870c-6978db8dcb33 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.892102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb966cc2-3fa6-4c57-870c-6978db8dcb33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.903293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb966cc2-3fa6-4c57-870c-6978db8dcb33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-eb966cc2-3fa6-4c57-870c-6978db8dcb33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:08.908171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:08.909054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:08.909848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 96/187] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:02:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:08.915379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2859-91df-4eaf-9845-5e7e6c9d443f None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.915977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2859-91df-4eaf-9845-5e7e6c9d443f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.916261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a2859-91df-4eaf-9845-5e7e6c9d443f None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.917614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 97/188] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:08.919186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a2859-91df-4eaf-9845-5e7e6c9d443f None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:08.943017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54f927-adc1-443f-953f-17a28d2b65ef None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:08.978472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf112b-6914-4345-8ed9-9200c1722e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:08.979127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf112b-6914-4345-8ed9-9200c1722e79 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:08.979400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf112b-6914-4345-8ed9-9200c1722e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:08.979662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf112b-6914-4345-8ed9-9200c1722e79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:08.990440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf112b-6914-4345-8ed9-9200c1722e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-3cbf112b-6914-4345-8ed9-9200c1722e79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:08.995142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:08.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:08.996852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 92/189] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:09.000321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e131be-b541-4c16-8d96-91a132f13894 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.000882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e131be-b541-4c16-8d96-91a132f13894 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.001357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e131be-b541-4c16-8d96-91a132f13894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.021554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-91e131be-b541-4c16-8d96-91a132f13894 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:09.022753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-91e131be-b541-4c16-8d96-91a132f13894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:09.034008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a2859-91df-4eaf-9845-5e7e6c9d443f None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.077140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.077757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.078033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.078774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.079713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:09.080162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:09.081531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:09.090515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 98/190] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:09.091337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:09.099785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 93/191] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:09.101072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e131be-b541-4c16-8d96-91a132f13894 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:09.107062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:09.123336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:09.202569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57c43c-de35-4182-b73b-1a00a8fca282 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.216227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e131be-b541-4c16-8d96-91a132f13894 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.236849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.237460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.237741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.238010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.238741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:09.239564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:09.240923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:09.253185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 99/192] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:09.253185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:09.257996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970f1a9-2a5f-4c0f-8b50-1bd4af174da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.258665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970f1a9-2a5f-4c0f-8b50-1bd4af174da8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.258941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970f1a9-2a5f-4c0f-8b50-1bd4af174da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.259208 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970f1a9-2a5f-4c0f-8b50-1bd4af174da8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.260308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970f1a9-2a5f-4c0f-8b50-1bd4af174da8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:09.261745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970f1a9-2a5f-4c0f-8b50-1bd4af174da8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:09.263369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 94/193] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:09.264187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970f1a9-2a5f-4c0f-8b50-1bd4af174da8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:09.339836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:09.348667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:09.363479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f88526-ad79-4d73-82db-95fddc89ea20 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.369796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970f1a9-2a5f-4c0f-8b50-1bd4af174da8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.398070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0bf55-edab-4554-ac58-0fb2e3e77c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.398735 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0bf55-edab-4554-ac58-0fb2e3e77c84 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.399012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0bf55-edab-4554-ac58-0fb2e3e77c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.399285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0bf55-edab-4554-ac58-0fb2e3e77c84 None admin] PATH_INFO: `/v3/users/9458423d410240f99243c22946ff957b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.404690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d37fe-c439-4eea-beb8-f97dd17aed2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.405308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d37fe-c439-4eea-beb8-f97dd17aed2d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.405579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d37fe-c439-4eea-beb8-f97dd17aed2d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.405843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d37fe-c439-4eea-beb8-f97dd17aed2d None admin] PATH_INFO: `/v3/users/9458423d410240f99243c22946ff957b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.413834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0bf55-edab-4554-ac58-0fb2e3e77c84 None admin] RBAC: Authorizing `identity:get_user(user_id=9458423d410240f99243c22946ff957b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:09.415782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0bf55-edab-4554-ac58-0fb2e3e77c84 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:09.418034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 100/194] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:09 2022] GET /identity/v3/users/9458423d410240f99243c22946ff957b => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:09.418823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0bf55-edab-4554-ac58-0fb2e3e77c84 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:09.419161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d37fe-c439-4eea-beb8-f97dd17aed2d None admin] RBAC: Authorizing `identity:get_user(user_id=9458423d410240f99243c22946ff957b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:09.420555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d37fe-c439-4eea-beb8-f97dd17aed2d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:09.422589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 95/195] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:09 2022] GET /identity/v3/users/9458423d410240f99243c22946ff957b => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:09.423370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d37fe-c439-4eea-beb8-f97dd17aed2d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:09.522777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0bf55-edab-4554-ac58-0fb2e3e77c84 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.524805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d37fe-c439-4eea-beb8-f97dd17aed2d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.557639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc0e3c-9e26-45ea-a2b6-8deca3db15e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.558129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f8f4-e731-4313-a5e3-0aade225861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.558253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc0e3c-9e26-45ea-a2b6-8deca3db15e3 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.558531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc0e3c-9e26-45ea-a2b6-8deca3db15e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.558787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f8f4-e731-4313-a5e3-0aade225861e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.558887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc0e3c-9e26-45ea-a2b6-8deca3db15e3 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.559063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f8f4-e731-4313-a5e3-0aade225861e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.559329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612f8f4-e731-4313-a5e3-0aade225861e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.559465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc0e3c-9e26-45ea-a2b6-8deca3db15e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:09.560902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc0e3c-9e26-45ea-a2b6-8deca3db15e3 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:09.565953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:09.569100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:09.570037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612f8f4-e731-4313-a5e3-0aade225861e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7612f8f4-e731-4313-a5e3-0aade225861e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:09.574943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:09.580968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 96/196] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:09.582194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f8f4-e731-4313-a5e3-0aade225861e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:09.631196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 101/197] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:09 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:09.632032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc0e3c-9e26-45ea-a2b6-8deca3db15e3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:09.683238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612f8f4-e731-4313-a5e3-0aade225861e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.723263 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2657c-42df-499e-8f00-af85c861350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.724285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2657c-42df-499e-8f00-af85c861350e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.724782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2657c-42df-499e-8f00-af85c861350e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.725271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2657c-42df-499e-8f00-af85c861350e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.753351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc0e3c-9e26-45ea-a2b6-8deca3db15e3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:09.790733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ee54d-4dc5-44dc-925a-55d951287087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:09.791297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ee54d-4dc5-44dc-925a-55d951287087 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:09.791581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ee54d-4dc5-44dc-925a-55d951287087 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:09.791847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ee54d-4dc5-44dc-925a-55d951287087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:09.809165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:09.813219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2657c-42df-499e-8f00-af85c861350e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:09.814832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2657c-42df-499e-8f00-af85c861350e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:09.858254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:09.870841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ee54d-4dc5-44dc-925a-55d951287087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:09.872613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8ee54d-4dc5-44dc-925a-55d951287087 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:09.941029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 97/198] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:09.942921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2657c-42df-499e-8f00-af85c861350e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:10.000406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 102/199] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:10.001663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ee54d-4dc5-44dc-925a-55d951287087 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:10.075881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:10.080989 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2657c-42df-499e-8f00-af85c861350e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:10.110408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:10.120822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ee54d-4dc5-44dc-925a-55d951287087 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:10.127894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:10.128512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:10.128793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:10.129068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:10.130886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:10.131196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:10.137222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:10.143594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 98/200] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:10.144956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:10.159559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855339-e3ec-405d-a1c6-538bc84f6484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:10.160604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855339-e3ec-405d-a1c6-538bc84f6484 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:10.161647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855339-e3ec-405d-a1c6-538bc84f6484 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:10.162252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855339-e3ec-405d-a1c6-538bc84f6484 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:10.163614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855339-e3ec-405d-a1c6-538bc84f6484 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:10.165411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855339-e3ec-405d-a1c6-538bc84f6484 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:10.167298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 103/201] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:10 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:10.168316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855339-e3ec-405d-a1c6-538bc84f6484 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:10.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404771c-7880-40b4-9c53-a917d7dac4e0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:10.297176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855339-e3ec-405d-a1c6-538bc84f6484 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:10.319175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984cf62-3dae-4e6a-b3b8-ad3919b2250d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:10.319814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984cf62-3dae-4e6a-b3b8-ad3919b2250d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:10.320091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984cf62-3dae-4e6a-b3b8-ad3919b2250d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:10.320366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984cf62-3dae-4e6a-b3b8-ad3919b2250d None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:10.322103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984cf62-3dae-4e6a-b3b8-ad3919b2250d None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:10.323932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7984cf62-3dae-4e6a-b3b8-ad3919b2250d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:10.326415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 99/202] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:10 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:10.327406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984cf62-3dae-4e6a-b3b8-ad3919b2250d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:10.338637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f01c33-d6e5-451d-9488-0f7dce4f3271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:10.339585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f01c33-d6e5-451d-9488-0f7dce4f3271 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:10.340053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f01c33-d6e5-451d-9488-0f7dce4f3271 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:10.340564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f01c33-d6e5-451d-9488-0f7dce4f3271 None admin] PATH_INFO: `/v3/users/7d8804b93f6b45e6bd4437b6c4272c23` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:10.341944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f01c33-d6e5-451d-9488-0f7dce4f3271 None admin] RBAC: Authorizing `identity:get_user(user_id=7d8804b93f6b45e6bd4437b6c4272c23)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:10.343594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f01c33-d6e5-451d-9488-0f7dce4f3271 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:10.345370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 104/203] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:10 2022] GET /identity/v3/users/7d8804b93f6b45e6bd4437b6c4272c23 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:10.346469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f01c33-d6e5-451d-9488-0f7dce4f3271 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:10.361941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:10.380345 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:10.469352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984cf62-3dae-4e6a-b3b8-ad3919b2250d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:10.475419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f01c33-d6e5-451d-9488-0f7dce4f3271 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:10.513813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:10.514615 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:10.514938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:10.515273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:10.516064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] RBAC: Adding query filter params (role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=9ca640bda24942c38dc244b54e934cf0, user.id=9458423d410240f99243c22946ff957b) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:10.516435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:10.518243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:10.519553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:10.521106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:10.521566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:10.522010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:10.523171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=9ca640bda24942c38dc244b54e934cf0, user.id=9458423d410240f99243c22946ff957b) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:10.523675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:10.525824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 105/204] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:10 2022] GET /identity/v3/role_assignments?user.id=9458423d410240f99243c22946ff957b&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:10.526465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:10.526814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:10.534686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 100/205] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:10 2022] GET /identity/v3/role_assignments?user.id=9458423d410240f99243c22946ff957b&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:10.643143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:10.656207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99514965-4836-4a12-bca3-77930947eed8 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:10.711848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f6ba3-eb14-4819-8b9a-f35afe1b6c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:10.713229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f6ba3-eb14-4819-8b9a-f35afe1b6c02 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:10.713229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f6ba3-eb14-4819-8b9a-f35afe1b6c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:10.713229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f6ba3-eb14-4819-8b9a-f35afe1b6c02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:10.810820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f6ba3-eb14-4819-8b9a-f35afe1b6c02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:10.812333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f6ba3-eb14-4819-8b9a-f35afe1b6c02 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:10.933842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 106/206] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:10.934986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:10.939513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:11.047050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:11.060764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f15b5-61f6-460c-8df7-49981bb8181f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:11.106642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512eb34c-fae9-4220-9c3e-1a8d1a0528bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:11.107550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eb34c-fae9-4220-9c3e-1a8d1a0528bf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:11.107982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eb34c-fae9-4220-9c3e-1a8d1a0528bf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:11.108469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512eb34c-fae9-4220-9c3e-1a8d1a0528bf None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:11.115236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512eb34c-fae9-4220-9c3e-1a8d1a0528bf None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:11.116965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512eb34c-fae9-4220-9c3e-1a8d1a0528bf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:11.118909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 101/207] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:10 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:11.125110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f6ba3-eb14-4819-8b9a-f35afe1b6c02 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:11.248739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f6ba3-eb14-4819-8b9a-f35afe1b6c02 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:11.289006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecafdb-622b-4ca5-aa73-7a61aea4ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:11.289892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecafdb-622b-4ca5-aa73-7a61aea4ce73 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:11.290364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecafdb-622b-4ca5-aa73-7a61aea4ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:11.290790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecafdb-622b-4ca5-aa73-7a61aea4ce73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:11.291703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecafdb-622b-4ca5-aa73-7a61aea4ce73 None admin] RBAC: Adding query filter params (role.id=be22e1353d404cea9407c96a9def8c8c, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e, user.id=7d8804b93f6b45e6bd4437b6c4272c23) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:11.292145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecafdb-622b-4ca5-aa73-7a61aea4ce73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:11.293750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecafdb-622b-4ca5-aa73-7a61aea4ce73 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:11.300476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 107/208] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:11 2022] GET /identity/v3/role_assignments?user.id=7d8804b93f6b45e6bd4437b6c4272c23&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=be22e1353d404cea9407c96a9def8c8c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:12.346962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20f190-9211-4be8-a2e3-c2e49eb36705 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:12.347988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20f190-9211-4be8-a2e3-c2e49eb36705 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:12.349048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20f190-9211-4be8-a2e3-c2e49eb36705 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:12.351047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 102/209] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:12.356334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17742b-4b14-4ae8-bfa6-3f9c9bb2c708 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:12.356819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17742b-4b14-4ae8-bfa6-3f9c9bb2c708 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:12.356892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17742b-4b14-4ae8-bfa6-3f9c9bb2c708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:12.376610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0a17742b-4b14-4ae8-bfa6-3f9c9bb2c708 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:12.377853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0a17742b-4b14-4ae8-bfa6-3f9c9bb2c708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:12.434280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:12.453367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 108/210] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:12.489103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a7967-3b4b-4bd1-b4eb-ae8e74a3beac None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:12.489530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a7967-3b4b-4bd1-b4eb-ae8e74a3beac None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:12.489787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a7967-3b4b-4bd1-b4eb-ae8e74a3beac None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:12.491411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 103/211] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:02:12.496940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4542184-f534-494e-8584-e8a2c9123d4e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:12.497286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4542184-f534-494e-8584-e8a2c9123d4e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:12.497561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4542184-f534-494e-8584-e8a2c9123d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:12.523295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c4542184-f534-494e-8584-e8a2c9123d4e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:12.524809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c4542184-f534-494e-8584-e8a2c9123d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:12.600336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 109/212] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:12.736981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a7967-3b4b-4bd1-b4eb-ae8e74a3beac None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:12.851422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a7967-3b4b-4bd1-b4eb-ae8e74a3beac None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:12.887274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5efb3-6d9b-4baf-9752-d3b5dc74887d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:12.887904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5efb3-6d9b-4baf-9752-d3b5dc74887d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:12.888180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5efb3-6d9b-4baf-9752-d3b5dc74887d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:12.888451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5efb3-6d9b-4baf-9752-d3b5dc74887d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:12.895581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4542184-f534-494e-8584-e8a2c9123d4e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:12.900984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5efb3-6d9b-4baf-9752-d3b5dc74887d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a2b5efb3-6d9b-4baf-9752-d3b5dc74887d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:12.906240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:12.907287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:12.908265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:12.910380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 104/213] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:12.918331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5efb3-6d9b-4baf-9752-d3b5dc74887d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:12.944748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:13.015684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4542184-f534-494e-8584-e8a2c9123d4e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:13.031125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:13.036179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5efb3-6d9b-4baf-9752-d3b5dc74887d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:13.053409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa9a19-e82b-4c2e-b60d-a3d18e8f4d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:13.054101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa9a19-e82b-4c2e-b60d-a3d18e8f4d96 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.054403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa9a19-e82b-4c2e-b60d-a3d18e8f4d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.054673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa9a19-e82b-4c2e-b60d-a3d18e8f4d96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.055779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa9a19-e82b-4c2e-b60d-a3d18e8f4d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:13.057445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa9a19-e82b-4c2e-b60d-a3d18e8f4d96 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:13.059367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 110/214] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:13.064482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa9a19-e82b-4c2e-b60d-a3d18e8f4d96 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:13.080217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05221540-c4b9-48a0-8184-0a2ca4ea75d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:13.081227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05221540-c4b9-48a0-8184-0a2ca4ea75d0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.081841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05221540-c4b9-48a0-8184-0a2ca4ea75d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.082311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05221540-c4b9-48a0-8184-0a2ca4ea75d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.083805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05221540-c4b9-48a0-8184-0a2ca4ea75d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:13.084381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05221540-c4b9-48a0-8184-0a2ca4ea75d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:13.086275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05221540-c4b9-48a0-8184-0a2ca4ea75d0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:13.095807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 105/215] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:13.097571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268a2b-f1c7-4951-90c7-e444cd3fead4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.098119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268a2b-f1c7-4951-90c7-e444cd3fead4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.098727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a268a2b-f1c7-4951-90c7-e444cd3fead4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.101288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 106/216] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:13.102939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a268a2b-f1c7-4951-90c7-e444cd3fead4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:13.184539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa9a19-e82b-4c2e-b60d-a3d18e8f4d96 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:13.203080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:13.223638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696296b6-b873-4a1c-ad33-c7770115dc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:13.227967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696296b6-b873-4a1c-ad33-c7770115dc4c None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.227967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696296b6-b873-4a1c-ad33-c7770115dc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.227967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696296b6-b873-4a1c-ad33-c7770115dc4c None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.227967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696296b6-b873-4a1c-ad33-c7770115dc4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:13.228275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a268a2b-f1c7-4951-90c7-e444cd3fead4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:13.228359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696296b6-b873-4a1c-ad33-c7770115dc4c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:13.263507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b6151-7a23-4797-9e21-677415d79592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:13.264134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6151-7a23-4797-9e21-677415d79592 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.264413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6151-7a23-4797-9e21-677415d79592 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.264686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b6151-7a23-4797-9e21-677415d79592 None admin] PATH_INFO: `/v3/users/b44054b792c24825b0d9a77acb4d22e5` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.278664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b6151-7a23-4797-9e21-677415d79592 None admin] RBAC: Authorizing `identity:get_user(user_id=b44054b792c24825b0d9a77acb4d22e5)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:13.280139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b6151-7a23-4797-9e21-677415d79592 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:13.283051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 107/217] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:13 2022] GET /identity/v3/users/b44054b792c24825b0d9a77acb4d22e5 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:13.283211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477604b-fd32-4f50-9b5a-9888ef6e5ef2 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.283444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477604b-fd32-4f50-9b5a-9888ef6e5ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.283708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d477604b-fd32-4f50-9b5a-9888ef6e5ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.301104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d477604b-fd32-4f50-9b5a-9888ef6e5ef2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:13.302426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d477604b-fd32-4f50-9b5a-9888ef6e5ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:13.330369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 111/218] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:13 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:02:13.330369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696296b6-b873-4a1c-ad33-c7770115dc4c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:13.375308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 108/219] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:13.376499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae91910-ba4f-4172-a771-54b25667efaf None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.376781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae91910-ba4f-4172-a771-54b25667efaf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.377047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae91910-ba4f-4172-a771-54b25667efaf None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.378399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 109/220] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:13.383418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:13.388137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270922c9-45a0-419e-82d0-026f18db1cde None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.388413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270922c9-45a0-419e-82d0-026f18db1cde None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.388724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270922c9-45a0-419e-82d0-026f18db1cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.419789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-270922c9-45a0-419e-82d0-026f18db1cde None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:13.424346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-270922c9-45a0-419e-82d0-026f18db1cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:13.477575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696296b6-b873-4a1c-ad33-c7770115dc4c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:13.516492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:13.526725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 110/221] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:13.528845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f52e2c-3b59-4ba0-b417-d2d25ba98a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:13.528845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f52e2c-3b59-4ba0-b417-d2d25ba98a6a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.528845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f52e2c-3b59-4ba0-b417-d2d25ba98a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.528845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f52e2c-3b59-4ba0-b417-d2d25ba98a6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.600446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f52e2c-3b59-4ba0-b417-d2d25ba98a6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:13.601896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f52e2c-3b59-4ba0-b417-d2d25ba98a6a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:13.640903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:13.655860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270922c9-45a0-419e-82d0-026f18db1cde None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:13.720030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 112/222] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:13.727550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f52e2c-3b59-4ba0-b417-d2d25ba98a6a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:13.799806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270922c9-45a0-419e-82d0-026f18db1cde None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:13.847412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ece757-db1e-485d-8e01-bfb66cb7e870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:13.848046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ece757-db1e-485d-8e01-bfb66cb7e870 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.848863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ece757-db1e-485d-8e01-bfb66cb7e870 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.849139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ece757-db1e-485d-8e01-bfb66cb7e870 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.862968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ece757-db1e-485d-8e01-bfb66cb7e870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-66ece757-db1e-485d-8e01-bfb66cb7e870 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:13.869687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:13.870622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:13.871437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:13.872026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 111/223] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:13.872754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ece757-db1e-485d-8e01-bfb66cb7e870 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:13.879968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f52e2c-3b59-4ba0-b417-d2d25ba98a6a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:13.924243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae18b3c-893e-4737-80ce-384679b2bdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:13.925291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18b3c-893e-4737-80ce-384679b2bdad None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:13.925570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18b3c-893e-4737-80ce-384679b2bdad None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:13.925985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18b3c-893e-4737-80ce-384679b2bdad None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:13.927635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae18b3c-893e-4737-80ce-384679b2bdad None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:13.929553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae18b3c-893e-4737-80ce-384679b2bdad None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:13.931246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 113/224] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:13 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:13.932752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae18b3c-893e-4737-80ce-384679b2bdad None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:13.938922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:13.954694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:13.991221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ece757-db1e-485d-8e01-bfb66cb7e870 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.039201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d501c4-a5f2-4c64-8577-64c29cdb80ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.039881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d501c4-a5f2-4c64-8577-64c29cdb80ba None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.040157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d501c4-a5f2-4c64-8577-64c29cdb80ba None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.040475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d501c4-a5f2-4c64-8577-64c29cdb80ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.054843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d501c4-a5f2-4c64-8577-64c29cdb80ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-32d501c4-a5f2-4c64-8577-64c29cdb80ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:14.061064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:14.061963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:14.062787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:14.069464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 112/225] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:02:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:14.069464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d501c4-a5f2-4c64-8577-64c29cdb80ba None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:14.070511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae18b3c-893e-4737-80ce-384679b2bdad None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.110885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.111494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.111774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.112045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.112879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:14.113237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:14.115321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:14.128758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 114/226] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:14.130362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:14.205116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d501c4-a5f2-4c64-8577-64c29cdb80ba None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.208345 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:14.232537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:14.247719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.248409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.248602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.248879 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.249864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] RBAC: Adding query filter params (role.id=453242bdc829422490366038c62c149a, scope.project.id=9ca640bda24942c38dc244b54e934cf0, user.id=b44054b792c24825b0d9a77acb4d22e5) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:14.250224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:14.251894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:14.257356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 113/227] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:14 2022] GET /identity/v3/role_assignments?user.id=b44054b792c24825b0d9a77acb4d22e5&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=453242bdc829422490366038c62c149a => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:14.258154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:14.263811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4d94a-2dd8-402f-a32f-d8f53c4cd82b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.319374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.321706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.321998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.322275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.323014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:14.323318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:14.324726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:14.335559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 115/228] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:14.341383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:14.473562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d48db9-c1de-4cb9-8dab-8ccc4d53a02f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.491567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba43ef5-3d5e-4447-b3d9-26d0cc6021c7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.508180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:14.514386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b14c-1851-4d81-b27d-b59bc5a24037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.515073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b14c-1851-4d81-b27d-b59bc5a24037 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.515357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b14c-1851-4d81-b27d-b59bc5a24037 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.515630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed1b14c-1851-4d81-b27d-b59bc5a24037 None admin] PATH_INFO: `/v3/users/9458423d410240f99243c22946ff957b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.517392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1b14c-1851-4d81-b27d-b59bc5a24037 None admin] RBAC: Authorizing `identity:get_user(user_id=9458423d410240f99243c22946ff957b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:14.519163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed1b14c-1851-4d81-b27d-b59bc5a24037 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:14.522466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 116/229] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:14 2022] GET /identity/v3/users/9458423d410240f99243c22946ff957b => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:14.527499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b14c-1851-4d81-b27d-b59bc5a24037 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:14.563030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78076419-b6c2-42a0-8e56-341fb7a8c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.564128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78076419-b6c2-42a0-8e56-341fb7a8c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.564605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78076419-b6c2-42a0-8e56-341fb7a8c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.565094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78076419-b6c2-42a0-8e56-341fb7a8c3c8 None admin] PATH_INFO: `/v3/users/9458423d410240f99243c22946ff957b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.571166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78076419-b6c2-42a0-8e56-341fb7a8c3c8 None admin] RBAC: Authorizing `identity:get_user(user_id=9458423d410240f99243c22946ff957b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:14.572892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78076419-b6c2-42a0-8e56-341fb7a8c3c8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:14.574784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 114/230] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:14 2022] GET /identity/v3/users/9458423d410240f99243c22946ff957b => generated 283 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:14.583198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78076419-b6c2-42a0-8e56-341fb7a8c3c8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:14.655312 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed1b14c-1851-4d81-b27d-b59bc5a24037 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.697131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13cc7d-e652-4293-91b7-4a8dffaf8e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.697917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13cc7d-e652-4293-91b7-4a8dffaf8e05 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.698310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13cc7d-e652-4293-91b7-4a8dffaf8e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.698716 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13cc7d-e652-4293-91b7-4a8dffaf8e05 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.700316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13cc7d-e652-4293-91b7-4a8dffaf8e05 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:14.701881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13cc7d-e652-4293-91b7-4a8dffaf8e05 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:14.704566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 117/231] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:14 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:14.708759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:14.712741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13cc7d-e652-4293-91b7-4a8dffaf8e05 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:14.802781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:14.823174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13cc7d-e652-4293-91b7-4a8dffaf8e05 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.836511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78076419-b6c2-42a0-8e56-341fb7a8c3c8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:14.872725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a81480-5c66-4f14-8b28-d07acd24b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.873392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81480-5c66-4f14-8b28-d07acd24b801 None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.873670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81480-5c66-4f14-8b28-d07acd24b801 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.873932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a81480-5c66-4f14-8b28-d07acd24b801 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0/users/9458423d410240f99243c22946ff957b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.875989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a81480-5c66-4f14-8b28-d07acd24b801 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca640bda24942c38dc244b54e934cf0, user_id=9458423d410240f99243c22946ff957b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:14.877433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a81480-5c66-4f14-8b28-d07acd24b801 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:14.898199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1817c8a-ffe4-444c-ba23-19e648413fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:14.904278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817c8a-ffe4-444c-ba23-19e648413fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:14.904278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817c8a-ffe4-444c-ba23-19e648413fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:14.904278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1817c8a-ffe4-444c-ba23-19e648413fb3 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:14.904278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1817c8a-ffe4-444c-ba23-19e648413fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:14.904278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1817c8a-ffe4-444c-ba23-19e648413fb3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:14.904278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 115/232] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:14 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:14.910041 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1817c8a-ffe4-444c-ba23-19e648413fb3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:14.963223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 118/233] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:02:14 2022] PUT /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0/users/9458423d410240f99243c22946ff957b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:15.058590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1817c8a-ffe4-444c-ba23-19e648413fb3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:15.086817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:15.107088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901d908-d016-40c6-90fb-3ad5221011f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:15.107766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d908-d016-40c6-90fb-3ad5221011f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:15.108044 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d908-d016-40c6-90fb-3ad5221011f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:15.108317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901d908-d016-40c6-90fb-3ad5221011f2 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0/users/9458423d410240f99243c22946ff957b/roles/54e2dc31e78b4516b2296086e9eb12c4` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:15.115335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901d908-d016-40c6-90fb-3ad5221011f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca640bda24942c38dc244b54e934cf0, user_id=9458423d410240f99243c22946ff957b, role_id=54e2dc31e78b4516b2296086e9eb12c4)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:15.116796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6901d908-d016-40c6-90fb-3ad5221011f2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:15.237747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 116/234] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:02:14 2022] PUT /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0/users/9458423d410240f99243c22946ff957b/roles/54e2dc31e78b4516b2296086e9eb12c4 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:16.149047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7ed6a-d8fb-4185-bfd2-57edf075287e None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.149563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7ed6a-d8fb-4185-bfd2-57edf075287e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.149637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df7ed6a-d8fb-4185-bfd2-57edf075287e None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.150981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 119/235] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:16.153031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94d1b7-70c6-43b8-ab8c-dfc401780aa7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.153416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94d1b7-70c6-43b8-ab8c-dfc401780aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.153573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94d1b7-70c6-43b8-ab8c-dfc401780aa7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.155066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 117/236] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:16.156130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806061b8-331d-4868-9945-5a71541d14a2 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.156403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806061b8-331d-4868-9945-5a71541d14a2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.156669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806061b8-331d-4868-9945-5a71541d14a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.162417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7348-2367-4e19-9330-9cd2f4664d03 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.162691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7348-2367-4e19-9330-9cd2f4664d03 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.164448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e7348-2367-4e19-9330-9cd2f4664d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.183863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-806061b8-331d-4868-9945-5a71541d14a2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.185061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-806061b8-331d-4868-9945-5a71541d14a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.203868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-063e7348-2367-4e19-9330-9cd2f4664d03 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.205310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-063e7348-2367-4e19-9330-9cd2f4664d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.319609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 120/237] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:16.329441 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 118/238] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:16.632002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806061b8-331d-4868-9945-5a71541d14a2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:16.712955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e7348-2367-4e19-9330-9cd2f4664d03 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:16.746804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:16.815166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806061b8-331d-4868-9945-5a71541d14a2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:16.851325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b03779-7149-473b-851c-d98a19a1f334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:16.851962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03779-7149-473b-851c-d98a19a1f334 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.852246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03779-7149-473b-851c-d98a19a1f334 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.852666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b03779-7149-473b-851c-d98a19a1f334 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.853792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b03779-7149-473b-851c-d98a19a1f334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:16.855329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b03779-7149-473b-851c-d98a19a1f334 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:16.856919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 121/239] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:16.858173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1160a98-5c41-418f-aac9-8795c1ca81cf None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.862773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1160a98-5c41-418f-aac9-8795c1ca81cf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.863084 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1160a98-5c41-418f-aac9-8795c1ca81cf None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.864470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 122/240] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:16.869061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1160a98-5c41-418f-aac9-8795c1ca81cf None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:16.869180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e7348-2367-4e19-9330-9cd2f4664d03 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:16.882736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:16.915391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fdd02-4be5-4913-a50a-188c1f10e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:16.916696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fdd02-4be5-4913-a50a-188c1f10e294 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.917483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fdd02-4be5-4913-a50a-188c1f10e294 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.917958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fdd02-4be5-4913-a50a-188c1f10e294 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.931654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fdd02-4be5-4913-a50a-188c1f10e294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-191fdd02-4be5-4913-a50a-188c1f10e294 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:16.938444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:16.939325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:16.940123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:16.943757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 119/241] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:16.948538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21d40a-f904-4030-96e3-510dc1a3a68d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:16.948538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21d40a-f904-4030-96e3-510dc1a3a68d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:16.948538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21d40a-f904-4030-96e3-510dc1a3a68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:16.970888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4c21d40a-f904-4030-96e3-510dc1a3a68d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:16.971989 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4c21d40a-f904-4030-96e3-510dc1a3a68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:16.991815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1160a98-5c41-418f-aac9-8795c1ca81cf None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.024534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.039540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb273-ceb2-450a-b908-b5557585a1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.040591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb273-ceb2-450a-b908-b5557585a1e0 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.041072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb273-ceb2-450a-b908-b5557585a1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.041562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb273-ceb2-450a-b908-b5557585a1e0 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.046392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 120/242] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:17.046392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21d40a-f904-4030-96e3-510dc1a3a68d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.047961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eb273-ceb2-450a-b908-b5557585a1e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.049999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eb273-ceb2-450a-b908-b5557585a1e0 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.157904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.162745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21d40a-f904-4030-96e3-510dc1a3a68d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.184925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 123/243] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:16 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:17.220496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.221448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.221872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.222282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.223689 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:17.224210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.225739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.238499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 121/244] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.244324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb273-ceb2-450a-b908-b5557585a1e0 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.374450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.380491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb273-ceb2-450a-b908-b5557585a1e0 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.466488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ac938-3d5f-4675-8393-96fb31c20d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.468318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ac938-3d5f-4675-8393-96fb31c20d80 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.468602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ac938-3d5f-4675-8393-96fb31c20d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.468875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ac938-3d5f-4675-8393-96fb31c20d80 None admin] PATH_INFO: `/v3/users/1b085b5d5775438e8482b5c3d9fe9bd0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.501359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ac938-3d5f-4675-8393-96fb31c20d80 None admin] RBAC: Authorizing `identity:get_user(user_id=1b085b5d5775438e8482b5c3d9fe9bd0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.502041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ac938-3d5f-4675-8393-96fb31c20d80 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.504246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 124/245] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/users/1b085b5d5775438e8482b5c3d9fe9bd0 => generated 297 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.511773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ac938-3d5f-4675-8393-96fb31c20d80 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.530283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c94e22-ae8f-4fc7-be8d-22e82623f6be None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.575607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.583475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0b0b6-9387-4d94-9636-627400f59d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.584234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0b0b6-9387-4d94-9636-627400f59d67 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.584588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0b0b6-9387-4d94-9636-627400f59d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.584935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0b0b6-9387-4d94-9636-627400f59d67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.597849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0b0b6-9387-4d94-9636-627400f59d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-72a0b0b6-9387-4d94-9636-627400f59d67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:17.605557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:17.607501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:17.608349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 122/246] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:17.613101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0b0b6-9387-4d94-9636-627400f59d67 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.645072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.658427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ac938-3d5f-4675-8393-96fb31c20d80 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.704593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a4de4-9a17-440a-a4f5-0bc6be2c598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.705613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4de4-9a17-440a-a4f5-0bc6be2c598a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.706971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4de4-9a17-440a-a4f5-0bc6be2c598a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.708357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a4de4-9a17-440a-a4f5-0bc6be2c598a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.750152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0b0b6-9387-4d94-9636-627400f59d67 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.794929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f7f6a-1a78-49ac-b17e-53c56ce7790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:17.795464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7f6a-1a78-49ac-b17e-53c56ce7790d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.795740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7f6a-1a78-49ac-b17e-53c56ce7790d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.796002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f7f6a-1a78-49ac-b17e-53c56ce7790d None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.796671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f7f6a-1a78-49ac-b17e-53c56ce7790d None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:17.796970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f7f6a-1a78-49ac-b17e-53c56ce7790d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.798370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f7f6a-1a78-49ac-b17e-53c56ce7790d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.804011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a4de4-9a17-440a-a4f5-0bc6be2c598a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:17.805778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a4de4-9a17-440a-a4f5-0bc6be2c598a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:17.810484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 123/247] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:17.810484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f257c-980b-4421-b34f-9be2d74a65f8 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.810484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f257c-980b-4421-b34f-9be2d74a65f8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.810484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f257c-980b-4421-b34f-9be2d74a65f8 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.811566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 124/248] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:17.815183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f257c-980b-4421-b34f-9be2d74a65f8 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:17.862432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.924330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:17.944724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 125/249] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:17.946024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308f1cd-0db0-49cd-b45c-b03bdd75cd11 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:17.946308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308f1cd-0db0-49cd-b45c-b03bdd75cd11 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:17.946628 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308f1cd-0db0-49cd-b45c-b03bdd75cd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:17.952015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f257c-980b-4421-b34f-9be2d74a65f8 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:17.971995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a308f1cd-0db0-49cd-b45c-b03bdd75cd11 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:17.973825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a308f1cd-0db0-49cd-b45c-b03bdd75cd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:18.009635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b158e-7bc9-4c1c-b80f-edf253c0f4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.010275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b158e-7bc9-4c1c-b80f-edf253c0f4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.010612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b158e-7bc9-4c1c-b80f-edf253c0f4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.010882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b158e-7bc9-4c1c-b80f-edf253c0f4e4 None admin] PATH_INFO: `/v3/users/b44054b792c24825b0d9a77acb4d22e5` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.012312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b158e-7bc9-4c1c-b80f-edf253c0f4e4 None admin] RBAC: Authorizing `identity:get_user(user_id=b44054b792c24825b0d9a77acb4d22e5)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.013768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b158e-7bc9-4c1c-b80f-edf253c0f4e4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.018993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 125/250] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:17 2022] GET /identity/v3/users/b44054b792c24825b0d9a77acb4d22e5 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:18.022824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b158e-7bc9-4c1c-b80f-edf253c0f4e4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.083059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 126/251] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:18.083938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308f1cd-0db0-49cd-b45c-b03bdd75cd11 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.154098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.157334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b158e-7bc9-4c1c-b80f-edf253c0f4e4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.189695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308f1cd-0db0-49cd-b45c-b03bdd75cd11 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.206248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb55470-1cf1-45a3-b7e8-5f8e80ca8ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.207156 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb55470-1cf1-45a3-b7e8-5f8e80ca8ecf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.207943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb55470-1cf1-45a3-b7e8-5f8e80ca8ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.208220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb55470-1cf1-45a3-b7e8-5f8e80ca8ecf None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.209571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb55470-1cf1-45a3-b7e8-5f8e80ca8ecf None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.211464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb55470-1cf1-45a3-b7e8-5f8e80ca8ecf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.213682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 126/252] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:18.214861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c843-0163-4fca-82b5-6ef8bd3f49b4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.215133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c843-0163-4fca-82b5-6ef8bd3f49b4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.215398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c843-0163-4fca-82b5-6ef8bd3f49b4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.216737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 127/253] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:18.221674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941c843-0163-4fca-82b5-6ef8bd3f49b4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.234816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7fbbf-4f08-424f-b25b-5f73db655db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.235511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fbbf-4f08-424f-b25b-5f73db655db5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.235788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fbbf-4f08-424f-b25b-5f73db655db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.236091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fbbf-4f08-424f-b25b-5f73db655db5 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.237319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7fbbf-4f08-424f-b25b-5f73db655db5 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.239360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7fbbf-4f08-424f-b25b-5f73db655db5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.241327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 127/254] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:18.242590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f25414-8e38-4ac7-a327-8155e73869ef None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.242892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f25414-8e38-4ac7-a327-8155e73869ef None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.243145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f25414-8e38-4ac7-a327-8155e73869ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.258553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.262044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-27f25414-8e38-4ac7-a327-8155e73869ef None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:18.263284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-27f25414-8e38-4ac7-a327-8155e73869ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:18.388181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 128/255] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:18.388815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f25414-8e38-4ac7-a327-8155e73869ef None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.403102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941c843-0163-4fca-82b5-6ef8bd3f49b4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.479910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.480758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.481193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.481594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.482472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] RBAC: Adding query filter params (role.id=be22e1353d404cea9407c96a9def8c8c, scope.project.id=9ca640bda24942c38dc244b54e934cf0, user.id=1b085b5d5775438e8482b5c3d9fe9bd0) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:18.482926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.484528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.497648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 128/256] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/role_assignments?user.id=1b085b5d5775438e8482b5c3d9fe9bd0&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=be22e1353d404cea9407c96a9def8c8c => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:18.498844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.533296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.616685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f25414-8e38-4ac7-a327-8155e73869ef None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.668065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:18.702932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d3618-a463-4a23-886e-27866ff0474f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.708626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf303f-d117-4468-8235-6419c8c8e83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.708626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf303f-d117-4468-8235-6419c8c8e83b None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.708626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf303f-d117-4468-8235-6419c8c8e83b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.708626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf303f-d117-4468-8235-6419c8c8e83b None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0/users/b44054b792c24825b0d9a77acb4d22e5/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.715219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf303f-d117-4468-8235-6419c8c8e83b None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca640bda24942c38dc244b54e934cf0, user_id=b44054b792c24825b0d9a77acb4d22e5, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:18.716687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf303f-d117-4468-8235-6419c8c8e83b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:18.773688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb4199-4e7d-44f8-b457-4c1e61471592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:18.774330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb4199-4e7d-44f8-b457-4c1e61471592 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:18.774630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb4199-4e7d-44f8-b457-4c1e61471592 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:18.774948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb4199-4e7d-44f8-b457-4c1e61471592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:18.787475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb4199-4e7d-44f8-b457-4c1e61471592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b6cb4199-4e7d-44f8-b457-4c1e61471592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:18.793238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:18.794154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 129/257] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:18.795952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb4199-4e7d-44f8-b457-4c1e61471592 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.800564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 129/258] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:02:18 2022] PUT /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0/users/b44054b792c24825b0d9a77acb4d22e5/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:18.807538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf303f-d117-4468-8235-6419c8c8e83b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:18.971251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf303f-d117-4468-8235-6419c8c8e83b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:18.991999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb4199-4e7d-44f8-b457-4c1e61471592 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.006783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.021872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.022528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.022807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.027113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.027113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:19.027113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.031443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.053854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 130/259] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:19.059654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.084726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712c36c-65be-4e7b-abd0-3f1e227c749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.085381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712c36c-65be-4e7b-abd0-3f1e227c749b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.085719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712c36c-65be-4e7b-abd0-3f1e227c749b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.085990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712c36c-65be-4e7b-abd0-3f1e227c749b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.121135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712c36c-65be-4e7b-abd0-3f1e227c749b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9712c36c-65be-4e7b-abd0-3f1e227c749b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:19.129753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:19.130767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:19.131550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:19.136186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 130/260] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:19.142000 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712c36c-65be-4e7b-abd0-3f1e227c749b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.148031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.227219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597882ab-d8b1-4b0c-9255-20ae517c3bb9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.263168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0112628-944d-40b3-b4d7-8a9d2662beb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.264151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0112628-944d-40b3-b4d7-8a9d2662beb8 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.264628 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0112628-944d-40b3-b4d7-8a9d2662beb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.265132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0112628-944d-40b3-b4d7-8a9d2662beb8 None admin] PATH_INFO: `/v3/users/9458423d410240f99243c22946ff957b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.266514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0112628-944d-40b3-b4d7-8a9d2662beb8 None admin] RBAC: Authorizing `identity:get_user(user_id=9458423d410240f99243c22946ff957b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.268179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0112628-944d-40b3-b4d7-8a9d2662beb8 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.274490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 131/261] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/users/9458423d410240f99243c22946ff957b => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:19.276236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0112628-944d-40b3-b4d7-8a9d2662beb8 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.329112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712c36c-65be-4e7b-abd0-3f1e227c749b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.335401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.381777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0112628-944d-40b3-b4d7-8a9d2662beb8 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.401315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.401928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.402203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.402511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.403170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:19.403463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.404848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.424228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 131/262] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:19.427814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc48de3-086c-43db-9b42-91265857b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.428661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc48de3-086c-43db-9b42-91265857b806 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.429293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc48de3-086c-43db-9b42-91265857b806 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.429786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc48de3-086c-43db-9b42-91265857b806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.430081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.501239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc48de3-086c-43db-9b42-91265857b806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.503043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc48de3-086c-43db-9b42-91265857b806 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.553763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.563382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.614525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 132/263] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:19.617523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc48de3-086c-43db-9b42-91265857b806 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.656341 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bef616-d93e-41d5-99fc-605578d8589a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.725808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc48de3-086c-43db-9b42-91265857b806 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.741154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747707ba-677d-4da4-89c6-75f2c1730ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.741868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747707ba-677d-4da4-89c6-75f2c1730ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.742147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747707ba-677d-4da4-89c6-75f2c1730ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.742443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747707ba-677d-4da4-89c6-75f2c1730ca9 None admin] PATH_INFO: `/v3/users/9458423d410240f99243c22946ff957b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.747066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747707ba-677d-4da4-89c6-75f2c1730ca9 None admin] RBAC: Authorizing `identity:get_user(user_id=9458423d410240f99243c22946ff957b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.748542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747707ba-677d-4da4-89c6-75f2c1730ca9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.750196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 132/264] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/users/9458423d410240f99243c22946ff957b => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:19.756197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747707ba-677d-4da4-89c6-75f2c1730ca9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.765859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05544003-d711-4b44-9550-3435afec0b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.766865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05544003-d711-4b44-9550-3435afec0b5b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.767349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05544003-d711-4b44-9550-3435afec0b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.767847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05544003-d711-4b44-9550-3435afec0b5b None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.769211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05544003-d711-4b44-9550-3435afec0b5b None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.771464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05544003-d711-4b44-9550-3435afec0b5b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.773415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 133/265] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:19.785794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05544003-d711-4b44-9550-3435afec0b5b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:19.818821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:19.899719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05544003-d711-4b44-9550-3435afec0b5b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:19.935495 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf2501-8230-49c8-a1e1-bd88d0367c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:19.936686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf2501-8230-49c8-a1e1-bd88d0367c9c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:19.937169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf2501-8230-49c8-a1e1-bd88d0367c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:19.937686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf2501-8230-49c8-a1e1-bd88d0367c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:19.938684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf2501-8230-49c8-a1e1-bd88d0367c9c None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=9ca640bda24942c38dc244b54e934cf0, user.id=9458423d410240f99243c22946ff957b) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:19.939207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf2501-8230-49c8-a1e1-bd88d0367c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:19.940907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf2501-8230-49c8-a1e1-bd88d0367c9c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:19.949481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 134/266] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/role_assignments?user.id=9458423d410240f99243c22946ff957b&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:19.988826 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747707ba-677d-4da4-89c6-75f2c1730ca9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.042174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9d855-c14d-4e4d-8312-f2a418d80f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.042725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9d855-c14d-4e4d-8312-f2a418d80f36 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.042998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9d855-c14d-4e4d-8312-f2a418d80f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.043266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9d855-c14d-4e4d-8312-f2a418d80f36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.048604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:20.144776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9d855-c14d-4e4d-8312-f2a418d80f36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.146366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9d855-c14d-4e4d-8312-f2a418d80f36 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.220319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04d35-03f5-4856-b137-a1a69bfe516c None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.220638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04d35-03f5-4856-b137-a1a69bfe516c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.220891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04d35-03f5-4856-b137-a1a69bfe516c None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.222486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 135/267] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:20.228157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44db95-7fdc-4cae-83e5-a1c1c53d05bf None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.228718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44db95-7fdc-4cae-83e5-a1c1c53d05bf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.228718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44db95-7fdc-4cae-83e5-a1c1c53d05bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.248243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-9a44db95-7fdc-4cae-83e5-a1c1c53d05bf None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:20.249412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9a44db95-7fdc-4cae-83e5-a1c1c53d05bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:20.286285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 133/268] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 23:02:20.292302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9d855-c14d-4e4d-8312-f2a418d80f36 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.333492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:20.337688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 136/269] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:20.401769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9d855-c14d-4e4d-8312-f2a418d80f36 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.437876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53501f8-0f82-498f-922f-7b5478febd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.438537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53501f8-0f82-498f-922f-7b5478febd45 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.438809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53501f8-0f82-498f-922f-7b5478febd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.439124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53501f8-0f82-498f-922f-7b5478febd45 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.440294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53501f8-0f82-498f-922f-7b5478febd45 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.441736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53501f8-0f82-498f-922f-7b5478febd45 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.443876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 134/270] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:20 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:20.449339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44db95-7fdc-4cae-83e5-a1c1c53d05bf None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.472423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:20.553684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44db95-7fdc-4cae-83e5-a1c1c53d05bf None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.591746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.592355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.592632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.592895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.593620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] RBAC: Adding query filter params (role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=9ca640bda24942c38dc244b54e934cf0, user.id=9458423d410240f99243c22946ff957b) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:20.593919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:20.595364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:20.602246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 137/271] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:20 2022] GET /identity/v3/role_assignments?user.id=9458423d410240f99243c22946ff957b&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:20.774091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53501f8-0f82-498f-922f-7b5478febd45 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:20.885797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53501f8-0f82-498f-922f-7b5478febd45 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:20.887285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:20.923661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fe241-0936-4fe9-ab4f-227b359d43cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:20.925010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fe241-0936-4fe9-ab4f-227b359d43cd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:20.925283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fe241-0936-4fe9-ab4f-227b359d43cd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:20.925551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fe241-0936-4fe9-ab4f-227b359d43cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:20.936393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675fe241-0936-4fe9-ab4f-227b359d43cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-675fe241-0936-4fe9-ab4f-227b359d43cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:20.941280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:20.942783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:20.943700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 135/272] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:20.950825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.055963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:21.076383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0473f66-63b6-4f0d-8ae9-7e229b1d8364 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:21.119156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cb07f-83be-4669-b27d-2561520aa05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:21.119766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cb07f-83be-4669-b27d-2561520aa05d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.120039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cb07f-83be-4669-b27d-2561520aa05d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.120307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cb07f-83be-4669-b27d-2561520aa05d None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.120965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cb07f-83be-4669-b27d-2561520aa05d None admin] RBAC: Adding query filter params (name=member) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:21.121263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cb07f-83be-4669-b27d-2561520aa05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:21.122710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1cb07f-83be-4669-b27d-2561520aa05d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:21.137837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 138/273] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:21.143893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fe241-0936-4fe9-ab4f-227b359d43cd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.252921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675fe241-0936-4fe9-ab4f-227b359d43cd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:21.271797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cf37f-0aa4-4fa5-8e69-b67671ac925f None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.274323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cf37f-0aa4-4fa5-8e69-b67671ac925f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.274323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cf37f-0aa4-4fa5-8e69-b67671ac925f None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.274323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 139/274] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:21.280727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce038de-c253-4072-b0dd-31a30c0a0e41 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.281238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce038de-c253-4072-b0dd-31a30c0a0e41 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.281707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce038de-c253-4072-b0dd-31a30c0a0e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.293506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41ce4d-5d6d-4edd-ada7-0af5b1a69bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:21.294131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41ce4d-5d6d-4edd-ada7-0af5b1a69bae None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.294432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41ce4d-5d6d-4edd-ada7-0af5b1a69bae None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.294701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41ce4d-5d6d-4edd-ada7-0af5b1a69bae None admin] PATH_INFO: `/v3/users/c07929316d70452fa11d610f2505c327` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.304000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0ce038de-c253-4072-b0dd-31a30c0a0e41 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:21.305741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0ce038de-c253-4072-b0dd-31a30c0a0e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:21.309625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41ce4d-5d6d-4edd-ada7-0af5b1a69bae None admin] RBAC: Authorizing `identity:get_user(user_id=c07929316d70452fa11d610f2505c327)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:21.311179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41ce4d-5d6d-4edd-ada7-0af5b1a69bae None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:21.313246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 136/275] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:21 2022] GET /identity/v3/users/c07929316d70452fa11d610f2505c327 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:21.319237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41ce4d-5d6d-4edd-ada7-0af5b1a69bae None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.345017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:21.398781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 140/276] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:21.451819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41ce4d-5d6d-4edd-ada7-0af5b1a69bae None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:21.494899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf06095-1208-4a7d-8fe1-39cf2f20ffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:21.495809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf06095-1208-4a7d-8fe1-39cf2f20ffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.496296 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf06095-1208-4a7d-8fe1-39cf2f20ffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.496747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf06095-1208-4a7d-8fe1-39cf2f20ffa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.497646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf06095-1208-4a7d-8fe1-39cf2f20ffa1 None admin] RBAC: Adding query filter params (role.id=453242bdc829422490366038c62c149a, user.id=c07929316d70452fa11d610f2505c327, scope.system=all) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:21.498126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf06095-1208-4a7d-8fe1-39cf2f20ffa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:21.499763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf06095-1208-4a7d-8fe1-39cf2f20ffa1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:21.510468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 137/277] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Sep 9 23:02:21 2022] GET /identity/v3/role_assignments?user.id=c07929316d70452fa11d610f2505c327&scope.system=all&role.id=453242bdc829422490366038c62c149a => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:21.580289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbed9c-9721-4bf4-96f0-719234f7b064 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.580563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbed9c-9721-4bf4-96f0-719234f7b064 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.580992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbed9c-9721-4bf4-96f0-719234f7b064 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.582949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 141/278] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:21.588967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482beae4-a890-401d-9739-941837cf7f2f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.589475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482beae4-a890-401d-9739-941837cf7f2f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.589886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482beae4-a890-401d-9739-941837cf7f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.610265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-482beae4-a890-401d-9739-941837cf7f2f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:21.611741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-482beae4-a890-401d-9739-941837cf7f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:21.691506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbed9c-9721-4bf4-96f0-719234f7b064 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.694718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 138/279] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:21.781139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:21.814108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fbed9c-9721-4bf4-96f0-719234f7b064 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:21.851114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8069af-6e55-4e52-8ced-b162e5d7f396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:21.851989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8069af-6e55-4e52-8ced-b162e5d7f396 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:21.852461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8069af-6e55-4e52-8ced-b162e5d7f396 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:21.852868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8069af-6e55-4e52-8ced-b162e5d7f396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:21.854166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8069af-6e55-4e52-8ced-b162e5d7f396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:21.855812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8069af-6e55-4e52-8ced-b162e5d7f396 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:21.857859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 142/280] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:21.863586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482beae4-a890-401d-9739-941837cf7f2f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.887065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:21.972603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8069af-6e55-4e52-8ced-b162e5d7f396 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:21.974517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482beae4-a890-401d-9739-941837cf7f2f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.020472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2ab1e-904d-428c-a842-f1355b4b3c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.021799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2ab1e-904d-428c-a842-f1355b4b3c76 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.022505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2ab1e-904d-428c-a842-f1355b4b3c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.022921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2ab1e-904d-428c-a842-f1355b4b3c76 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.023766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2ab1e-904d-428c-a842-f1355b4b3c76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.026318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2ab1e-904d-428c-a842-f1355b4b3c76 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:22.080813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8069af-6e55-4e52-8ced-b162e5d7f396 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.115553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400a5ce-4779-4600-93b9-00b3180d4590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.116174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400a5ce-4779-4600-93b9-00b3180d4590 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.116453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400a5ce-4779-4600-93b9-00b3180d4590 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.116722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400a5ce-4779-4600-93b9-00b3180d4590 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.126524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400a5ce-4779-4600-93b9-00b3180d4590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-f400a5ce-4779-4600-93b9-00b3180d4590 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:22.132360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:22.133231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:22.134051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:22.135406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 139/281] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:21 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:22.135406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c0df4-88bb-44e0-973a-b0d87893c549 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.138390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 143/282] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:22.139256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400a5ce-4779-4600-93b9-00b3180d4590 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.142164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:22.142292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c0df4-88bb-44e0-973a-b0d87893c549 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.142663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c0df4-88bb-44e0-973a-b0d87893c549 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.149708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 140/283] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:22.151440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5f05e-e3ce-4530-a771-cc159f3c0b67 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.152091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5f05e-e3ce-4530-a771-cc159f3c0b67 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.152366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5f05e-e3ce-4530-a771-cc159f3c0b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.207416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6ad5f05e-e3ce-4530-a771-cc159f3c0b67 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:22.208700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6ad5f05e-e3ce-4530-a771-cc159f3c0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:22.253680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400a5ce-4779-4600-93b9-00b3180d4590 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.297647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.298279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.298583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.298951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.299727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:22.300031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.301699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:22.314520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 144/284] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:22.319006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.321027 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:22.327376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 141/285] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:22.389595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:22.428326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa592cdd-2906-4968-adda-cd7a6031149e None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.464520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948448a-e3ad-4760-8046-afebc6a038e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.465588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948448a-e3ad-4760-8046-afebc6a038e4 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.466045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948448a-e3ad-4760-8046-afebc6a038e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.466322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948448a-e3ad-4760-8046-afebc6a038e4 None admin] PATH_INFO: `/v3/users/1b085b5d5775438e8482b5c3d9fe9bd0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.467674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948448a-e3ad-4760-8046-afebc6a038e4 None admin] RBAC: Authorizing `identity:get_user(user_id=1b085b5d5775438e8482b5c3d9fe9bd0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.469168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948448a-e3ad-4760-8046-afebc6a038e4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:22.470926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 145/286] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:22 2022] GET /identity/v3/users/1b085b5d5775438e8482b5c3d9fe9bd0 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:22.477440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5f05e-e3ce-4530-a771-cc159f3c0b67 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.586574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5f05e-e3ce-4530-a771-cc159f3c0b67 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.605292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948448a-e3ad-4760-8046-afebc6a038e4 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.633314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb7f0d-09de-4906-8b75-fd39fd899d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.633543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7f0d-09de-4906-8b75-fd39fd899d23 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.633746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7f0d-09de-4906-8b75-fd39fd899d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.634021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb7f0d-09de-4906-8b75-fd39fd899d23 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.635607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb7f0d-09de-4906-8b75-fd39fd899d23 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.637600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb7f0d-09de-4906-8b75-fd39fd899d23 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:22.640072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 142/287] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:22 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:22.644897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb7f0d-09de-4906-8b75-fd39fd899d23 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.740969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:22.749468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948448a-e3ad-4760-8046-afebc6a038e4 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.808165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb7f0d-09de-4906-8b75-fd39fd899d23 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:22.814104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:22.824838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a8723-1dda-4cc7-9d63-1fd2cac267d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.833126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a8723-1dda-4cc7-9d63-1fd2cac267d2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.833126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a8723-1dda-4cc7-9d63-1fd2cac267d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.833126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a8723-1dda-4cc7-9d63-1fd2cac267d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.848109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a8723-1dda-4cc7-9d63-1fd2cac267d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-616a8723-1dda-4cc7-9d63-1fd2cac267d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:22.860383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:22.861294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:22.862090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:22.863231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 146/288] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:22.874375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a8723-1dda-4cc7-9d63-1fd2cac267d2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:22.897575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd8a88-2732-4257-abfd-76a9982a225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:22.898246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd8a88-2732-4257-abfd-76a9982a225b None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:22.898546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd8a88-2732-4257-abfd-76a9982a225b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:22.898870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd8a88-2732-4257-abfd-76a9982a225b None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0/users/1b085b5d5775438e8482b5c3d9fe9bd0/roles/be22e1353d404cea9407c96a9def8c8c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:22.902073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd8a88-2732-4257-abfd-76a9982a225b None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca640bda24942c38dc244b54e934cf0, user_id=1b085b5d5775438e8482b5c3d9fe9bd0, role_id=be22e1353d404cea9407c96a9def8c8c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:22.904319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd8a88-2732-4257-abfd-76a9982a225b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:23.011144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 143/289] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:02:22 2022] PUT /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0/users/1b085b5d5775438e8482b5c3d9fe9bd0/roles/be22e1353d404cea9407c96a9def8c8c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:02:23.025979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a8723-1dda-4cc7-9d63-1fd2cac267d2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:23.077159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:23.077792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.078068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.078336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.079065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:23.079631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:23.081043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:23.093661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 147/290] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:23.100317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd8a88-2732-4257-abfd-76a9982a225b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:23.231618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd8a88-2732-4257-abfd-76a9982a225b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:23.264904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:23.273704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c143f1-906a-465c-b468-6a05f19cf55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:23.274365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c143f1-906a-465c-b468-6a05f19cf55d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.274654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c143f1-906a-465c-b468-6a05f19cf55d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.274932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c143f1-906a-465c-b468-6a05f19cf55d None admin] PATH_INFO: `/v3/users/b44054b792c24825b0d9a77acb4d22e5` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.276161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c143f1-906a-465c-b468-6a05f19cf55d None admin] RBAC: Authorizing `identity:get_user(user_id=b44054b792c24825b0d9a77acb4d22e5)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:23.277614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c143f1-906a-465c-b468-6a05f19cf55d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:23.279661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 144/291] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:23 2022] GET /identity/v3/users/b44054b792c24825b0d9a77acb4d22e5 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:23.285769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:23.346597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:23.393860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e752454-f1ff-47e6-9e39-c058a16a90d2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:23.451632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1610b4-8e83-46e5-8dd2-adfc2ee75527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:23.452263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1610b4-8e83-46e5-8dd2-adfc2ee75527 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.452544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1610b4-8e83-46e5-8dd2-adfc2ee75527 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.452821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1610b4-8e83-46e5-8dd2-adfc2ee75527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.538196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1610b4-8e83-46e5-8dd2-adfc2ee75527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:23.539738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1610b4-8e83-46e5-8dd2-adfc2ee75527 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:23.600244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:23.648590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 148/292] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:23.657761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c143f1-906a-465c-b468-6a05f19cf55d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:23.784975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c143f1-906a-465c-b468-6a05f19cf55d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:23.835166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36216f-0876-42cb-ac9d-908a5895714d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:23.835837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36216f-0876-42cb-ac9d-908a5895714d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.836119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36216f-0876-42cb-ac9d-908a5895714d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.836397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36216f-0876-42cb-ac9d-908a5895714d None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.837721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36216f-0876-42cb-ac9d-908a5895714d None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:23.839369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36216f-0876-42cb-ac9d-908a5895714d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:23.841318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 145/293] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:23 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:23.847563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1610b4-8e83-46e5-8dd2-adfc2ee75527 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:23.943794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5149-15c7-4f92-87e3-24cc03db3303 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.944085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5149-15c7-4f92-87e3-24cc03db3303 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.944265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c5149-15c7-4f92-87e3-24cc03db3303 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.945677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 146/294] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:23.950563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9748-9112-48b8-8ac5-ca28089b9481 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:23.950850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9748-9112-48b8-8ac5-ca28089b9481 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:23.951127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee9748-9112-48b8-8ac5-ca28089b9481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:23.982600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-00ee9748-9112-48b8-8ac5-ca28089b9481 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:23.983804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-00ee9748-9112-48b8-8ac5-ca28089b9481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:24.000520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1610b4-8e83-46e5-8dd2-adfc2ee75527 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:24.057163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:24.058230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.058959 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.059271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.060010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] RBAC: Adding query filter params (role.id=453242bdc829422490366038c62c149a, scope.project.id=9ca640bda24942c38dc244b54e934cf0, user.id=b44054b792c24825b0d9a77acb4d22e5) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:24.060434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:24.062054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:24.069770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 149/295] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:23 2022] GET /identity/v3/role_assignments?user.id=b44054b792c24825b0d9a77acb4d22e5&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=453242bdc829422490366038c62c149a => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:24.082006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:24.103520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 147/296] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:24.475495 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:24.502767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:24.595750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba623ab-9ea2-47ee-8f30-7b8cc982165d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:24.636767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c9544-78c6-4d8b-9c7f-ecccf46ebd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:24.637826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c9544-78c6-4d8b-9c7f-ecccf46ebd1f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.638306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c9544-78c6-4d8b-9c7f-ecccf46ebd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.638974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c9544-78c6-4d8b-9c7f-ecccf46ebd1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.653822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c9544-78c6-4d8b-9c7f-ecccf46ebd1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-2b3c9544-78c6-4d8b-9c7f-ecccf46ebd1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:24.659121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:24.659964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:24.661309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:24.663827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 150/297] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:24.669518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee9748-9112-48b8-8ac5-ca28089b9481 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:24.779219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee9748-9112-48b8-8ac5-ca28089b9481 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:24.818047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:24.818693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.818941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.819202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.819857 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:24.820242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:24.821522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:24.830856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 148/298] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:24.836898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c9544-78c6-4d8b-9c7f-ecccf46ebd1f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:24.917246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:24.938990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c9544-78c6-4d8b-9c7f-ecccf46ebd1f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:24.975388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f680a-1f07-40ef-adec-7a6b6d7e706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:24.976066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f680a-1f07-40ef-adec-7a6b6d7e706d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:24.976345 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f680a-1f07-40ef-adec-7a6b6d7e706d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:24.976612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f680a-1f07-40ef-adec-7a6b6d7e706d None admin] PATH_INFO: `/v3/users/c07929316d70452fa11d610f2505c327` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:24.977743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f680a-1f07-40ef-adec-7a6b6d7e706d None admin] RBAC: Authorizing `identity:get_user(user_id=c07929316d70452fa11d610f2505c327)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:24.979204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f680a-1f07-40ef-adec-7a6b6d7e706d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:24.981081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 151/299] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:24 2022] GET /identity/v3/users/c07929316d70452fa11d610f2505c327 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:24.986742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.050466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:25.105207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a18d9-a865-488e-8441-6aef184b3fd8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:25.140567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdcfeff-bc0d-40bf-8807-e3070951e50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:25.141223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcfeff-bc0d-40bf-8807-e3070951e50f None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.141498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcfeff-bc0d-40bf-8807-e3070951e50f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.141769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdcfeff-bc0d-40bf-8807-e3070951e50f None admin] PATH_INFO: `/v3/system/users/c07929316d70452fa11d610f2505c327/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.143217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdcfeff-bc0d-40bf-8807-e3070951e50f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c07929316d70452fa11d610f2505c327, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:25.144620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdcfeff-bc0d-40bf-8807-e3070951e50f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:25.178212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f4837-927f-4604-9505-1e8649415380 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.178593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f4837-927f-4604-9505-1e8649415380 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.178861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f4837-927f-4604-9505-1e8649415380 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.180258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 152/300] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:02:25.181268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329788-033b-491e-ba25-6e50c915d638 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.181639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329788-033b-491e-ba25-6e50c915d638 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.181914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329788-033b-491e-ba25-6e50c915d638 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.183162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 153/301] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:25.185413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c401a-5900-48a1-9ed6-1ec1d1eaa511 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.185690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c401a-5900-48a1-9ed6-1ec1d1eaa511 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.185951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c401a-5900-48a1-9ed6-1ec1d1eaa511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.204097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-848c401a-5900-48a1-9ed6-1ec1d1eaa511 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:25.205554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-848c401a-5900-48a1-9ed6-1ec1d1eaa511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:25.225602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 149/302] 10.222.0.20 () {58 vars in 1334 bytes} [Fri Sep 9 23:02:24 2022] PUT /identity/v3/system/users/c07929316d70452fa11d610f2505c327/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:25.235927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574807-a9f7-4cca-9913-2f9f9affe7f7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.236562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574807-a9f7-4cca-9913-2f9f9affe7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.236984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12574807-a9f7-4cca-9913-2f9f9affe7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.259448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-12574807-a9f7-4cca-9913-2f9f9affe7f7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:25.261154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-12574807-a9f7-4cca-9913-2f9f9affe7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:25.306161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 154/303] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:25.353635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 150/304] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:25.572619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a759db-9a9c-4308-86fd-09183c0e02b6 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.573101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a759db-9a9c-4308-86fd-09183c0e02b6 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.573181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a759db-9a9c-4308-86fd-09183c0e02b6 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.574780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 155/305] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:25.577263 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12574807-a9f7-4cca-9913-2f9f9affe7f7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.581368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba552-f60c-4f90-bd74-d7cba93ef0ac None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.581639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba552-f60c-4f90-bd74-d7cba93ef0ac None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.581903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaba552-f60c-4f90-bd74-d7cba93ef0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.607259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-1aaba552-f60c-4f90-bd74-d7cba93ef0ac None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:25.608597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1aaba552-f60c-4f90-bd74-d7cba93ef0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:25.650968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:25.686402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 156/306] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:25.686402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaba552-f60c-4f90-bd74-d7cba93ef0ac None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.695941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:25.709314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12574807-a9f7-4cca-9913-2f9f9affe7f7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:25.745665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52cd20f-dbfb-411b-b2ee-e861975329e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:25.746679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cd20f-dbfb-411b-b2ee-e861975329e6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.747150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cd20f-dbfb-411b-b2ee-e861975329e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.747670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52cd20f-dbfb-411b-b2ee-e861975329e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.758879 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52cd20f-dbfb-411b-b2ee-e861975329e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a52cd20f-dbfb-411b-b2ee-e861975329e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:25.764088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:25.764949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:25.765811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:25.768496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 151/307] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:25.777847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52cd20f-dbfb-411b-b2ee-e861975329e6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.795467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaba552-f60c-4f90-bd74-d7cba93ef0ac None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:25.841972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc46bf-da28-4777-ae89-017dace92f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:25.842555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc46bf-da28-4777-ae89-017dace92f3e None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.842831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc46bf-da28-4777-ae89-017dace92f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.843103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc46bf-da28-4777-ae89-017dace92f3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.844291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc46bf-da28-4777-ae89-017dace92f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:25.845766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc46bf-da28-4777-ae89-017dace92f3e None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:25.847660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 157/308] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:25.853245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc46bf-da28-4777-ae89-017dace92f3e None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.885235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52cd20f-dbfb-411b-b2ee-e861975329e6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:25.898688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:25.922981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:25.923831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:25.924683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:25.924967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:25.925914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:25.926282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:25.928076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:25.934621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:25.940329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 152/309] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:25.947323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:25.966100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc46bf-da28-4777-ae89-017dace92f3e None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.011166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd4e56-eb63-40de-8c07-b2bcff27c75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.011774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4e56-eb63-40de-8c07-b2bcff27c75a None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.012051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4e56-eb63-40de-8c07-b2bcff27c75a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.012319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd4e56-eb63-40de-8c07-b2bcff27c75a None admin] PATH_INFO: `/v3/groups` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.012935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd4e56-eb63-40de-8c07-b2bcff27c75a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.014586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd4e56-eb63-40de-8c07-b2bcff27c75a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.059890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c3fe5-dfc6-480f-8c21-95a72a91daa9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.095745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854a7c8-2e86-420b-b4be-cf78bbb060fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.096292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854a7c8-2e86-420b-b4be-cf78bbb060fc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.096566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854a7c8-2e86-420b-b4be-cf78bbb060fc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.096837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854a7c8-2e86-420b-b4be-cf78bbb060fc None admin] PATH_INFO: `/v3/users/dcdfedf9756b4481b7e72623288c6854` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.100170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 158/310] 10.222.0.20 () {60 vars in 1134 bytes} [Fri Sep 9 23:02:25 2022] POST /identity/v3/groups => generated 229 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:26.107453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd4e56-eb63-40de-8c07-b2bcff27c75a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:26.119047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854a7c8-2e86-420b-b4be-cf78bbb060fc None admin] RBAC: Authorizing `identity:get_user(user_id=dcdfedf9756b4481b7e72623288c6854)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.121731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854a7c8-2e86-420b-b4be-cf78bbb060fc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.125276 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 153/311] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:25 2022] GET /identity/v3/users/dcdfedf9756b4481b7e72623288c6854 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:26.132954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854a7c8-2e86-420b-b4be-cf78bbb060fc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:26.178431 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:26.247688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd4e56-eb63-40de-8c07-b2bcff27c75a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.278587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854a7c8-2e86-420b-b4be-cf78bbb060fc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.293288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75400e-aaa2-4a77-a71b-2cb17f44b07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.293940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75400e-aaa2-4a77-a71b-2cb17f44b07b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.294218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75400e-aaa2-4a77-a71b-2cb17f44b07b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.294520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75400e-aaa2-4a77-a71b-2cb17f44b07b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.296890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:26.307507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75400e-aaa2-4a77-a71b-2cb17f44b07b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-8d75400e-aaa2-4a77-a71b-2cb17f44b07b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:26.313352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:26.314250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:26.315436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 159/312] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:26.321586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75400e-aaa2-4a77-a71b-2cb17f44b07b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:26.328013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.328615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.328888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.329154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.330143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] RBAC: Adding query filter params (role.id=be22e1353d404cea9407c96a9def8c8c, user.id=dcdfedf9756b4481b7e72623288c6854, scope.system=all) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:26.330484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.331908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.340102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 154/313] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Sep 9 23:02:26 2022] GET /identity/v3/role_assignments?user.id=dcdfedf9756b4481b7e72623288c6854&scope.system=all&role.id=be22e1353d404cea9407c96a9def8c8c => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:26.464516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75400e-aaa2-4a77-a71b-2cb17f44b07b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.503469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.504145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.504496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.504959 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.505737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:26.506043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.507505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.517463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 160/314] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:26.524009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:26.625884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:26.633077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2076f-397d-4917-87c6-0aaf5ba68f98 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.668079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9d474-ee40-4a85-907a-68dd4aa51e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.668631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d474-ee40-4a85-907a-68dd4aa51e9f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.668908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d474-ee40-4a85-907a-68dd4aa51e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.669182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9d474-ee40-4a85-907a-68dd4aa51e9f None admin] PATH_INFO: `/v3/users/1b085b5d5775438e8482b5c3d9fe9bd0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.670321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9d474-ee40-4a85-907a-68dd4aa51e9f None admin] RBAC: Authorizing `identity:get_user(user_id=1b085b5d5775438e8482b5c3d9fe9bd0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.671795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9d474-ee40-4a85-907a-68dd4aa51e9f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.673568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 155/315] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:26 2022] GET /identity/v3/users/1b085b5d5775438e8482b5c3d9fe9bd0 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:26.679197 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:26.745121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:26.798112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d8fa7-cc77-4f19-8f16-b09d8fd6288c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:26.834506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb77265-9a0a-4e4a-8c87-e35d52964c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:26.835118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb77265-9a0a-4e4a-8c87-e35d52964c71 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:26.835611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb77265-9a0a-4e4a-8c87-e35d52964c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:26.835889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb77265-9a0a-4e4a-8c87-e35d52964c71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:26.905284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb77265-9a0a-4e4a-8c87-e35d52964c71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:26.906784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb77265-9a0a-4e4a-8c87-e35d52964c71 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:26.964071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:27.009742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 161/316] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:27.015399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9d474-ee40-4a85-907a-68dd4aa51e9f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:27.125119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9d474-ee40-4a85-907a-68dd4aa51e9f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:27.160471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039e194d-e808-47e6-9225-bd5f59a6bf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:27.161146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e194d-e808-47e6-9225-bd5f59a6bf4b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:27.161432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e194d-e808-47e6-9225-bd5f59a6bf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:27.161701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039e194d-e808-47e6-9225-bd5f59a6bf4b None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:27.162931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e194d-e808-47e6-9225-bd5f59a6bf4b None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:27.164411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039e194d-e808-47e6-9225-bd5f59a6bf4b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:27.166149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 156/317] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:27 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:27.172474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb77265-9a0a-4e4a-8c87-e35d52964c71 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:27.282704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb77265-9a0a-4e4a-8c87-e35d52964c71 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:27.318601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a551a-3450-4902-ba5f-c0776caa4d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:27.319202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a551a-3450-4902-ba5f-c0776caa4d65 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:27.319478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a551a-3450-4902-ba5f-c0776caa4d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:27.319751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a551a-3450-4902-ba5f-c0776caa4d65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:27.320468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a551a-3450-4902-ba5f-c0776caa4d65 None admin] RBAC: Adding query filter params (role.id=be22e1353d404cea9407c96a9def8c8c, scope.project.id=9ca640bda24942c38dc244b54e934cf0, user.id=1b085b5d5775438e8482b5c3d9fe9bd0) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:27.320773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a551a-3450-4902-ba5f-c0776caa4d65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:27.322192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a551a-3450-4902-ba5f-c0776caa4d65 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:27.328283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 162/318] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:27 2022] GET /identity/v3/role_assignments?user.id=1b085b5d5775438e8482b5c3d9fe9bd0&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=be22e1353d404cea9407c96a9def8c8c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:27.498666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b856b8-068c-44f6-bc2f-7ea0dd716217 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:27.499004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b856b8-068c-44f6-bc2f-7ea0dd716217 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:27.499207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b856b8-068c-44f6-bc2f-7ea0dd716217 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:27.500555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 157/319] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:27.506690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f52838-f913-40a6-8346-b0b1fc035642 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:27.506977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f52838-f913-40a6-8346-b0b1fc035642 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:27.507248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f52838-f913-40a6-8346-b0b1fc035642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:27.529074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-28f52838-f913-40a6-8346-b0b1fc035642 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:27.530266 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-28f52838-f913-40a6-8346-b0b1fc035642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:27.549890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:27.612748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 163/320] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:27.877787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b856b8-068c-44f6-bc2f-7ea0dd716217 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:27.911298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:27.987635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b856b8-068c-44f6-bc2f-7ea0dd716217 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:28.021208 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae02b75-27a2-4a2e-87db-1d7f6c5f05fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:28.021831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae02b75-27a2-4a2e-87db-1d7f6c5f05fe None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:28.022081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae02b75-27a2-4a2e-87db-1d7f6c5f05fe None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:28.022363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae02b75-27a2-4a2e-87db-1d7f6c5f05fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:28.032330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae02b75-27a2-4a2e-87db-1d7f6c5f05fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-1ae02b75-27a2-4a2e-87db-1d7f6c5f05fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:28.037818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:28.038897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:28.039696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 158/321] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:28.044636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f52838-f913-40a6-8346-b0b1fc035642 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:28.144561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f52838-f913-40a6-8346-b0b1fc035642 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:28.178510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:28.179114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:28.179387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:28.179653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:28.180300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:28.180652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:28.181997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:28.191365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 164/322] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:28.196537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae02b75-27a2-4a2e-87db-1d7f6c5f05fe None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:28.297843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:28.307425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae02b75-27a2-4a2e-87db-1d7f6c5f05fe None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:28.342649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f0501-3c8d-4be4-8807-0df662ef8e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:28.343273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f0501-3c8d-4be4-8807-0df662ef8e34 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:28.343544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f0501-3c8d-4be4-8807-0df662ef8e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:28.343809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f0501-3c8d-4be4-8807-0df662ef8e34 None admin] PATH_INFO: `/v3/users/c07929316d70452fa11d610f2505c327` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:28.344908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f0501-3c8d-4be4-8807-0df662ef8e34 None admin] RBAC: Authorizing `identity:get_user(user_id=c07929316d70452fa11d610f2505c327)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:28.346355 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f0501-3c8d-4be4-8807-0df662ef8e34 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:28.348078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 159/323] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:28 2022] GET /identity/v3/users/c07929316d70452fa11d610f2505c327 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:28.353625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:28.368771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:28.468884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335cfedc-cc83-4897-a85e-841c2fcd9db3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:28.503114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43caf591-5747-4f5d-929d-2c0c2f09a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:28.503707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caf591-5747-4f5d-929d-2c0c2f09a04d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:28.503979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caf591-5747-4f5d-929d-2c0c2f09a04d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:28.504288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43caf591-5747-4f5d-929d-2c0c2f09a04d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:28.505027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43caf591-5747-4f5d-929d-2c0c2f09a04d None admin] RBAC: Adding query filter params (role.id=453242bdc829422490366038c62c149a, user.id=c07929316d70452fa11d610f2505c327, scope.system=all) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:28.505324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43caf591-5747-4f5d-929d-2c0c2f09a04d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:28.506784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43caf591-5747-4f5d-929d-2c0c2f09a04d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:28.518444 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 165/324] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Sep 9 23:02:28 2022] GET /identity/v3/role_assignments?user.id=c07929316d70452fa11d610f2505c327&scope.system=all&role.id=453242bdc829422490366038c62c149a => generated 510 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:28.558226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e6570-5932-4367-b61a-92a6910a4b6b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:28.558604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e6570-5932-4367-b61a-92a6910a4b6b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:28.558817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e6570-5932-4367-b61a-92a6910a4b6b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:28.560392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 160/325] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:28.562854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9102c94-de02-4e12-9b21-7e0ed1ea3da0 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:28.563141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9102c94-de02-4e12-9b21-7e0ed1ea3da0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:28.563412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9102c94-de02-4e12-9b21-7e0ed1ea3da0 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:28.569497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 166/326] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:28.569663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412e332-3c06-403a-9113-d07d6aa2f035 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:28.569663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412e332-3c06-403a-9113-d07d6aa2f035 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:28.569663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412e332-3c06-403a-9113-d07d6aa2f035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:28.570952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d990b2-f1aa-4efc-9c48-0dfaffdd34cf None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:28.571230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d990b2-f1aa-4efc-9c48-0dfaffdd34cf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:28.571497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d990b2-f1aa-4efc-9c48-0dfaffdd34cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:28.590116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1412e332-3c06-403a-9113-d07d6aa2f035 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:28.590500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-43d990b2-f1aa-4efc-9c48-0dfaffdd34cf None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:28.591378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-43d990b2-f1aa-4efc-9c48-0dfaffdd34cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:28.591473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1412e332-3c06-403a-9113-d07d6aa2f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:28.661116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:28.666565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 161/327] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:28.666750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 167/328] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:28.939636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412e332-3c06-403a-9113-d07d6aa2f035 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:28.957067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d990b2-f1aa-4efc-9c48-0dfaffdd34cf None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:29.025681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:29.037379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412e332-3c06-403a-9113-d07d6aa2f035 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.058385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d990b2-f1aa-4efc-9c48-0dfaffdd34cf None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.072559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf178a-eb56-4bdb-ae4e-a0e1587e9613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.073188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf178a-eb56-4bdb-ae4e-a0e1587e9613 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.073471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf178a-eb56-4bdb-ae4e-a0e1587e9613 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.073803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf178a-eb56-4bdb-ae4e-a0e1587e9613 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.074947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf178a-eb56-4bdb-ae4e-a0e1587e9613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:29.076379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf178a-eb56-4bdb-ae4e-a0e1587e9613 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:29.077949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 162/329] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:29.084236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf178a-eb56-4bdb-ae4e-a0e1587e9613 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:29.093338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258b4fb-4a07-421f-b349-462e3ebb260c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.093964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258b4fb-4a07-421f-b349-462e3ebb260c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.094238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258b4fb-4a07-421f-b349-462e3ebb260c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.094536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258b4fb-4a07-421f-b349-462e3ebb260c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.104459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258b4fb-4a07-421f-b349-462e3ebb260c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-c258b4fb-4a07-421f-b349-462e3ebb260c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:29.109219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:29.110130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:29.110942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 168/330] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:29.116052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258b4fb-4a07-421f-b349-462e3ebb260c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:29.185152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:29.190818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf178a-eb56-4bdb-ae4e-a0e1587e9613 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.228969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258b4fb-4a07-421f-b349-462e3ebb260c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.229622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802d16-da60-413d-a63e-fb51b6824083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.231533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802d16-da60-413d-a63e-fb51b6824083 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.231533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802d16-da60-413d-a63e-fb51b6824083 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.231533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802d16-da60-413d-a63e-fb51b6824083 None admin] PATH_INFO: `/v3/groups` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.231533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e802d16-da60-413d-a63e-fb51b6824083 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:29.232883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e802d16-da60-413d-a63e-fb51b6824083 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:29.264088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.264705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.264978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.265243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.265886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:29.266178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:29.267585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:29.276223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 169/331] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:29.281870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:29.305456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 163/332] 10.222.0.20 () {60 vars in 1134 bytes} [Fri Sep 9 23:02:29 2022] POST /identity/v3/groups => generated 226 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:02:29.393548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a1fd8-aab3-4a3f-a422-81ef28e80026 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.419281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:29.431133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30cf92-e411-4ca6-a3c1-cefeb0a33fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.431781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30cf92-e411-4ca6-a3c1-cefeb0a33fee None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.432055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30cf92-e411-4ca6-a3c1-cefeb0a33fee None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.432323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30cf92-e411-4ca6-a3c1-cefeb0a33fee None admin] PATH_INFO: `/v3/users/dcdfedf9756b4481b7e72623288c6854` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.433490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30cf92-e411-4ca6-a3c1-cefeb0a33fee None admin] RBAC: Authorizing `identity:get_user(user_id=dcdfedf9756b4481b7e72623288c6854)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:29.434992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30cf92-e411-4ca6-a3c1-cefeb0a33fee None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:29.436579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 170/333] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:29 2022] GET /identity/v3/users/dcdfedf9756b4481b7e72623288c6854 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:29.442986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802d16-da60-413d-a63e-fb51b6824083 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:29.463525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:29.552540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802d16-da60-413d-a63e-fb51b6824083 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:29.590666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176d70c-4f15-41cf-bbd9-facf90f00799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:29.591311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176d70c-4f15-41cf-bbd9-facf90f00799 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:29.591586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176d70c-4f15-41cf-bbd9-facf90f00799 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:29.591853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176d70c-4f15-41cf-bbd9-facf90f00799 None admin] PATH_INFO: `/v3/system/users/dcdfedf9756b4481b7e72623288c6854/roles/be22e1353d404cea9407c96a9def8c8c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:29.593269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176d70c-4f15-41cf-bbd9-facf90f00799 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dcdfedf9756b4481b7e72623288c6854, role_id=be22e1353d404cea9407c96a9def8c8c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:29.594674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176d70c-4f15-41cf-bbd9-facf90f00799 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:29.659886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 164/334] 10.222.0.20 () {58 vars in 1334 bytes} [Fri Sep 9 23:02:29 2022] PUT /identity/v3/system/users/dcdfedf9756b4481b7e72623288c6854/roles/be22e1353d404cea9407c96a9def8c8c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:32.017906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf6d14-f14f-4342-b524-83a6b390ec15 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.018522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf6d14-f14f-4342-b524-83a6b390ec15 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.018522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf6d14-f14f-4342-b524-83a6b390ec15 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.019975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 171/335] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:32.025914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7357-ca1a-4272-a4b5-5a9288a75e14 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.026439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7357-ca1a-4272-a4b5-5a9288a75e14 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.026512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b7357-ca1a-4272-a4b5-5a9288a75e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.047117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c58b7357-ca1a-4272-a4b5-5a9288a75e14 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:32.050793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c58b7357-ca1a-4272-a4b5-5a9288a75e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:32.132536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 165/336] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:32.193966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236baf79-5429-40ca-9020-0b15482d5a3c None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.194268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236baf79-5429-40ca-9020-0b15482d5a3c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.194550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236baf79-5429-40ca-9020-0b15482d5a3c None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.196153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 172/337] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:02:32.208235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac939-e955-42f6-bcf8-9b6925e815cc None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.208518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac939-e955-42f6-bcf8-9b6925e815cc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.208794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ac939-e955-42f6-bcf8-9b6925e815cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.235829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f53ac939-e955-42f6-bcf8-9b6925e815cc None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:32.237042 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f53ac939-e955-42f6-bcf8-9b6925e815cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:32.243535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:32.317644 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 166/338] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:32.607780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236baf79-5429-40ca-9020-0b15482d5a3c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.688105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b20fde-f4e6-4bbf-ab96-a7d1abed562d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.688446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b20fde-f4e6-4bbf-ab96-a7d1abed562d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.688662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b20fde-f4e6-4bbf-ab96-a7d1abed562d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.694031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 167/339] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:32.701959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b5e49-601e-4359-9f3c-a62a999806ac None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.702243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b5e49-601e-4359-9f3c-a62a999806ac None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.708028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b5e49-601e-4359-9f3c-a62a999806ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.741335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236baf79-5429-40ca-9020-0b15482d5a3c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:32.744846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-936b5e49-601e-4359-9f3c-a62a999806ac None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:32.745423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-936b5e49-601e-4359-9f3c-a62a999806ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:32.788201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9865b-d21a-4c33-a67b-ecfb69efd2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:32.789177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9865b-d21a-4c33-a67b-ecfb69efd2ae None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.789627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9865b-d21a-4c33-a67b-ecfb69efd2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.790066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9865b-d21a-4c33-a67b-ecfb69efd2ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.804441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9865b-d21a-4c33-a67b-ecfb69efd2ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-23f9865b-d21a-4c33-a67b-ecfb69efd2ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:32.810700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:32.811583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 09 23:02:32.812368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:32.818090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 173/340] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:32.820036 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0799c-e818-4ed1-83a3-6b79fcc6b935 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.820533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0799c-e818-4ed1-83a3-6b79fcc6b935 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.821001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0799c-e818-4ed1-83a3-6b79fcc6b935 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.823019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 174/341] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:32.824029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0799c-e818-4ed1-83a3-6b79fcc6b935 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.857332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 168/342] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:32.858812 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eeb03-afd9-4abc-b95b-003d02b48003 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:32.859048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eeb03-afd9-4abc-b95b-003d02b48003 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:32.859319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eeb03-afd9-4abc-b95b-003d02b48003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:32.878624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0b7eeb03-afd9-4abc-b95b-003d02b48003 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:32.880977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0b7eeb03-afd9-4abc-b95b-003d02b48003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:32.893544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:32.957808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:32.963927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 169/343] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:32.964974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eeb03-afd9-4abc-b95b-003d02b48003 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:32.965164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0799c-e818-4ed1-83a3-6b79fcc6b935 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.006438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.007051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.007331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.007599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.008625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:33.008931 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.010491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.021976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 175/344] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:33.033126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.169433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eeb03-afd9-4abc-b95b-003d02b48003 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.260599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ed9e3-5709-4d81-b751-afe8f27c3d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.261246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ed9e3-5709-4d81-b751-afe8f27c3d91 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.261521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ed9e3-5709-4d81-b751-afe8f27c3d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.261790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ed9e3-5709-4d81-b751-afe8f27c3d91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.280475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ed9e3-5709-4d81-b751-afe8f27c3d91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:33.284484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b0f6a-9893-4402-a963-07402256a911 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-8e1ed9e3-5709-4d81-b751-afe8f27c3d91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:33.286158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:33.288541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:33.290544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 170/345] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:33.299539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ed9e3-5709-4d81-b751-afe8f27c3d91 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.311016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.370800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dfdf-d4d9-4607-a8d9-a7b2e2f7c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.371496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dfdf-d4d9-4607-a8d9-a7b2e2f7c754 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.371785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dfdf-d4d9-4607-a8d9-a7b2e2f7c754 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.372058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dfdf-d4d9-4607-a8d9-a7b2e2f7c754 None admin] PATH_INFO: `/v3/users/dcdfedf9756b4481b7e72623288c6854` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.373215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90dfdf-d4d9-4607-a8d9-a7b2e2f7c754 None admin] RBAC: Authorizing `identity:get_user(user_id=dcdfedf9756b4481b7e72623288c6854)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.378975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90dfdf-d4d9-4607-a8d9-a7b2e2f7c754 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.381980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 176/346] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:02:33 2022] GET /identity/v3/users/dcdfedf9756b4481b7e72623288c6854 => generated 293 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:33.387969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dfdf-d4d9-4607-a8d9-a7b2e2f7c754 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.406877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.472207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ed9e3-5709-4d81-b751-afe8f27c3d91 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.539826 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.544201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.544201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.544201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.544201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:33.544201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.544201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.553562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 171/347] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:33.554177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.580784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dfdf-d4d9-4607-a8d9-a7b2e2f7c754 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.625376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.626374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.626862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.627253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.627980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] RBAC: Adding query filter params (role.id=be22e1353d404cea9407c96a9def8c8c, user.id=dcdfedf9756b4481b7e72623288c6854, scope.system=all) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:33.628434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:33.631330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:33.646331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 177/348] 10.222.0.20 () {56 vars in 1308 bytes} [Fri Sep 9 23:02:33 2022] GET /identity/v3/role_assignments?user.id=dcdfedf9756b4481b7e72623288c6854&scope.system=all&role.id=be22e1353d404cea9407c96a9def8c8c => generated 510 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:33.647212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.680020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ee78e-ac16-4c96-bbf6-fd60815409be None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.698985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.711622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.725203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d06235-24f4-4bea-930e-5b652d42f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.726236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06235-24f4-4bea-930e-5b652d42f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.726771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06235-24f4-4bea-930e-5b652d42f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.727212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d06235-24f4-4bea-930e-5b652d42f5d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.741090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d06235-24f4-4bea-930e-5b652d42f5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f1d06235-24f4-4bea-930e-5b652d42f5d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:33.748649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:33.750986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:33.751961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:33.753471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 172/349] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:33.754948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d06235-24f4-4bea-930e-5b652d42f5d6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.776517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d9e7e-1582-4369-82d7-207dd998a3d2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.816663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5724a81-00b3-4147-8b33-314b431c90b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.818017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5724a81-00b3-4147-8b33-314b431c90b5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.818295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5724a81-00b3-4147-8b33-314b431c90b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.818589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5724a81-00b3-4147-8b33-314b431c90b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.830479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5724a81-00b3-4147-8b33-314b431c90b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-c5724a81-00b3-4147-8b33-314b431c90b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:33.836771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:33.837857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:33.838709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 178/350] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:33.842894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5724a81-00b3-4147-8b33-314b431c90b5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:33.880788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d06235-24f4-4bea-930e-5b652d42f5d6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.922116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1dc6d-5ee5-4730-b4ab-c4b910c1e7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:33.923126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1dc6d-5ee5-4730-b4ab-c4b910c1e7ca None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:33.923646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1dc6d-5ee5-4730-b4ab-c4b910c1e7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:33.924186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1dc6d-5ee5-4730-b4ab-c4b910c1e7ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:33.956525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:33.962381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5724a81-00b3-4147-8b33-314b431c90b5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:33.995789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.012366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.013545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.013982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.014433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.021519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] RBAC: Adding query filter params (name=member) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:34.021676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1dc6d-5ee5-4730-b4ab-c4b910c1e7ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.021765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.021765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.021911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1dc6d-5ee5-4730-b4ab-c4b910c1e7ca None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.028383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 179/351] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:34.029752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.151619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92a6c6-70c2-43f3-b2e6-0618ead58a70 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.155940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 173/352] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:34.157225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9f44-0a02-4534-a151-df3aa2a2e0a5 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.157508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9f44-0a02-4534-a151-df3aa2a2e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.157778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9f44-0a02-4534-a151-df3aa2a2e0a5 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.159216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 174/353] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:34.159774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f9f44-0a02-4534-a151-df3aa2a2e0a5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.190598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04d4ca-6f4a-4ca4-ae69-a5bf957450ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.191204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04d4ca-6f4a-4ca4-ae69-a5bf957450ec None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.191478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04d4ca-6f4a-4ca4-ae69-a5bf957450ec None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.192561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04d4ca-6f4a-4ca4-ae69-a5bf957450ec None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.193226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04d4ca-6f4a-4ca4-ae69-a5bf957450ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:34.193522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04d4ca-6f4a-4ca4-ae69-a5bf957450ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.194918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04d4ca-6f4a-4ca4-ae69-a5bf957450ec None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.205010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 180/354] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:34.206332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b8d98-09a7-43dc-94e1-d95cc8f05602 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.206708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b8d98-09a7-43dc-94e1-d95cc8f05602 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.206965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b8d98-09a7-43dc-94e1-d95cc8f05602 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.209379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 181/355] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:34.210040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b8d98-09a7-43dc-94e1-d95cc8f05602 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.241043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.260957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.281892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f9f44-0a02-4534-a151-df3aa2a2e0a5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.325570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406367f-d706-4dbd-95fe-ef30de769dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.326445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406367f-d706-4dbd-95fe-ef30de769dfa None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.327027 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406367f-d706-4dbd-95fe-ef30de769dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.327301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406367f-d706-4dbd-95fe-ef30de769dfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.333542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b8d98-09a7-43dc-94e1-d95cc8f05602 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.380689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52402307-b5a7-4ad2-ae6a-31d335025f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.381323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402307-b5a7-4ad2-ae6a-31d335025f3f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.381649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402307-b5a7-4ad2-ae6a-31d335025f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.381872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52402307-b5a7-4ad2-ae6a-31d335025f3f None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.383056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402307-b5a7-4ad2-ae6a-31d335025f3f None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.384487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52402307-b5a7-4ad2-ae6a-31d335025f3f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.386142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 182/356] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:34.388250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f55a-bb27-4536-85dd-8ce70dc4d087 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.388528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f55a-bb27-4536-85dd-8ce70dc4d087 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.388802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753f55a-bb27-4536-85dd-8ce70dc4d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.407815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e753f55a-bb27-4536-85dd-8ce70dc4d087 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:34.408120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406367f-d706-4dbd-95fe-ef30de769dfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.409007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e753f55a-bb27-4536-85dd-8ce70dc4d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:34.409556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406367f-d706-4dbd-95fe-ef30de769dfa None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.489685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 183/357] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:34.490890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753f55a-bb27-4536-85dd-8ce70dc4d087 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.509376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.515493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.542127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 175/358] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:34.542504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478cdc-ccb4-408d-ada7-c296b96ec789 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.543808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478cdc-ccb4-408d-ada7-c296b96ec789 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.544083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab478cdc-ccb4-408d-ada7-c296b96ec789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.567285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ab478cdc-ccb4-408d-ada7-c296b96ec789 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:34.571310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ab478cdc-ccb4-408d-ada7-c296b96ec789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:34.620037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753f55a-bb27-4536-85dd-8ce70dc4d087 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.652075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 176/359] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:34.652935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab478cdc-ccb4-408d-ada7-c296b96ec789 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.661738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09fbc3-7f5f-4107-82c4-325be4ab5bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.662675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09fbc3-7f5f-4107-82c4-325be4ab5bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.663145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09fbc3-7f5f-4107-82c4-325be4ab5bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.663626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09fbc3-7f5f-4107-82c4-325be4ab5bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.741722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09fbc3-7f5f-4107-82c4-325be4ab5bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.744025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09fbc3-7f5f-4107-82c4-325be4ab5bc2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.765999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab478cdc-ccb4-408d-ada7-c296b96ec789 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:34.768637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.782019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:34.819505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642d20f-99ba-4c93-8cb6-aa32b9cc2f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:34.820152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642d20f-99ba-4c93-8cb6-aa32b9cc2f2d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:34.820426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642d20f-99ba-4c93-8cb6-aa32b9cc2f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:34.820693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642d20f-99ba-4c93-8cb6-aa32b9cc2f2d None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:34.831242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642d20f-99ba-4c93-8cb6-aa32b9cc2f2d None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:34.834523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b642d20f-99ba-4c93-8cb6-aa32b9cc2f2d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:34.836476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 177/360] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:34.837439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642d20f-99ba-4c93-8cb6-aa32b9cc2f2d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.882380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 184/361] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:34.882380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09fbc3-7f5f-4107-82c4-325be4ab5bc2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:34.971409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b642d20f-99ba-4c93-8cb6-aa32b9cc2f2d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.006391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09fbc3-7f5f-4107-82c4-325be4ab5bc2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.012366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ec2d5-3703-4d3c-82ed-1bc114206588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.013697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ec2d5-3703-4d3c-82ed-1bc114206588 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.014003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ec2d5-3703-4d3c-82ed-1bc114206588 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.014283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ec2d5-3703-4d3c-82ed-1bc114206588 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.015893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ec2d5-3703-4d3c-82ed-1bc114206588 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.017260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ec2d5-3703-4d3c-82ed-1bc114206588 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.019101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 178/362] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.019703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ec2d5-3703-4d3c-82ed-1bc114206588 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.028365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.045573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330075-7341-42e4-983f-5bcbc37ad0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.046306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330075-7341-42e4-983f-5bcbc37ad0da None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.046512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330075-7341-42e4-983f-5bcbc37ad0da None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.046771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330075-7341-42e4-983f-5bcbc37ad0da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.061640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1330075-7341-42e4-983f-5bcbc37ad0da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-b1330075-7341-42e4-983f-5bcbc37ad0da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:35.068552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:35.069442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:35.070235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 185/363] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:35.071038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.071112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330075-7341-42e4-983f-5bcbc37ad0da None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.136874 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ec2d5-3703-4d3c-82ed-1bc114206588 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.177169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.177775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.178056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.178328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.180535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] RBAC: Adding query filter params (group.id=261c9390f4f3487a8261852bc911f851, role.id=453242bdc829422490366038c62c149a, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:35.180854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.182299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.189276 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 179/364] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/role_assignments?group.id=261c9390f4f3487a8261852bc911f851&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=453242bdc829422490366038c62c149a => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.194572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.194779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330075-7341-42e4-983f-5bcbc37ad0da None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.245665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea222a-c610-4442-b7d1-5ccd29de9072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.246317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea222a-c610-4442-b7d1-5ccd29de9072 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.247123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea222a-c610-4442-b7d1-5ccd29de9072 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.247123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea222a-c610-4442-b7d1-5ccd29de9072 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.248714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea222a-c610-4442-b7d1-5ccd29de9072 None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.250424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea222a-c610-4442-b7d1-5ccd29de9072 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.252225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 186/365] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.253031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea222a-c610-4442-b7d1-5ccd29de9072 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.327816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.347673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a40a30-ed06-4b9e-a514-6722b977a5c3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.360770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.390862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409944c-b99f-4dfe-a6b8-ffcb42ed02bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.395491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409944c-b99f-4dfe-a6b8-ffcb42ed02bf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.395491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409944c-b99f-4dfe-a6b8-ffcb42ed02bf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.395491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409944c-b99f-4dfe-a6b8-ffcb42ed02bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.399532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea222a-c610-4442-b7d1-5ccd29de9072 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.405182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409944c-b99f-4dfe-a6b8-ffcb42ed02bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e409944c-b99f-4dfe-a6b8-ffcb42ed02bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:35.413661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:35.414615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:35.415485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 180/366] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:35.416625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409944c-b99f-4dfe-a6b8-ffcb42ed02bf None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.454115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981e6b4-5744-4244-9259-58afcbd9e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.455173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981e6b4-5744-4244-9259-58afcbd9e113 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.455449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981e6b4-5744-4244-9259-58afcbd9e113 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.455715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981e6b4-5744-4244-9259-58afcbd9e113 None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.459073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981e6b4-5744-4244-9259-58afcbd9e113 None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.460299 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981e6b4-5744-4244-9259-58afcbd9e113 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.462081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 187/367] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.462742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981e6b4-5744-4244-9259-58afcbd9e113 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.559385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409944c-b99f-4dfe-a6b8-ffcb42ed02bf None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.601188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981e6b4-5744-4244-9259-58afcbd9e113 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.612626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.613569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.614501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.614985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.616418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:35.616418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.617838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.627294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 181/368] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.627741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.628267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.655251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106c4d4-5ba9-4c69-94d5-a4dfc03e1db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.655891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c4d4-5ba9-4c69-94d5-a4dfc03e1db2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.659945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c4d4-5ba9-4c69-94d5-a4dfc03e1db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.660266 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106c4d4-5ba9-4c69-94d5-a4dfc03e1db2 None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.662024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106c4d4-5ba9-4c69-94d5-a4dfc03e1db2 None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.663522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106c4d4-5ba9-4c69-94d5-a4dfc03e1db2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.666814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 188/369] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.667782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106c4d4-5ba9-4c69-94d5-a4dfc03e1db2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.670532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.761511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe674a3-0489-40b4-b7ee-7d5aa39f66d4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.805359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.805826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106c4d4-5ba9-4c69-94d5-a4dfc03e1db2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:35.805960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.806219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.806518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.807173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:35.807475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.808836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.821805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 182/370] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.822736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.850623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:35.851406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:35.851786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:35.852153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:35.852876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] RBAC: Adding query filter params (group.id=261c9390f4f3487a8261852bc911f851, role.id=453242bdc829422490366038c62c149a, scope.project.id=9ca640bda24942c38dc244b54e934cf0) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:35.853185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:35.854651 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:35.866465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 189/371] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/role_assignments?group.id=261c9390f4f3487a8261852bc911f851&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=453242bdc829422490366038c62c149a => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:35.867477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:35.927602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.954227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:35.966467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d9d3f-324f-4d08-a127-d83ca2c65d37 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.000525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6789cb-5bed-4366-a2cd-d6e50f51c4d9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.016411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da15261-8898-4cda-83cf-eaf6c605018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.017040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da15261-8898-4cda-83cf-eaf6c605018e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.017322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da15261-8898-4cda-83cf-eaf6c605018e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.017592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da15261-8898-4cda-83cf-eaf6c605018e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.048461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.050157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.050658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.051279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.053254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] RBAC: Adding query filter params (group.id=261c9390f4f3487a8261852bc911f851, role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:36.054131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.056051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.064192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 190/372] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/role_assignments?group.id=261c9390f4f3487a8261852bc911f851&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:36.066077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.102261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da15261-8898-4cda-83cf-eaf6c605018e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.103734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da15261-8898-4cda-83cf-eaf6c605018e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.265338 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:36.269438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607ae1a-9a31-4ff9-bee5-75c38edd5d57 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.278059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 183/373] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:36.289967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da15261-8898-4cda-83cf-eaf6c605018e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.292700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:36.316240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7a735-e209-4b61-878e-854a00e161aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.317121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7a735-e209-4b61-878e-854a00e161aa None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.317407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7a735-e209-4b61-878e-854a00e161aa None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.317678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7a735-e209-4b61-878e-854a00e161aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.426628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da15261-8898-4cda-83cf-eaf6c605018e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.441076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7a735-e209-4b61-878e-854a00e161aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.442857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7a735-e209-4b61-878e-854a00e161aa None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.487153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeee891-d9dd-4414-8deb-2e74e57cc667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.487781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeee891-d9dd-4414-8deb-2e74e57cc667 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.488053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeee891-d9dd-4414-8deb-2e74e57cc667 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.488319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeee891-d9dd-4414-8deb-2e74e57cc667 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.490768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeee891-d9dd-4414-8deb-2e74e57cc667 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.492222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeee891-d9dd-4414-8deb-2e74e57cc667 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.495123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 184/374] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:36.499102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeee891-d9dd-4414-8deb-2e74e57cc667 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.602714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:36.635099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 191/375] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:36.641553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7a735-e209-4b61-878e-854a00e161aa None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.660132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:36.662096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeee891-d9dd-4414-8deb-2e74e57cc667 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.702118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104afb1b-42ce-43e3-804d-7bf89c730ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.702700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104afb1b-42ce-43e3-804d-7bf89c730ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.702974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104afb1b-42ce-43e3-804d-7bf89c730ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.703238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104afb1b-42ce-43e3-804d-7bf89c730ac2 None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.707047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104afb1b-42ce-43e3-804d-7bf89c730ac2 None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.708818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104afb1b-42ce-43e3-804d-7bf89c730ac2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.710718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 185/376] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:36.716649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104afb1b-42ce-43e3-804d-7bf89c730ac2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.776300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7a735-e209-4b61-878e-854a00e161aa None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.819785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabec5f-527b-43a9-aea0-5de628fb919a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.820436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabec5f-527b-43a9-aea0-5de628fb919a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.820711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabec5f-527b-43a9-aea0-5de628fb919a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.820983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabec5f-527b-43a9-aea0-5de628fb919a None admin] PATH_INFO: `/v3/projects/7393fe084b62454e9d96e9d57737a145` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.829541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabec5f-527b-43a9-aea0-5de628fb919a None admin] RBAC: Authorizing `identity:get_project(project_id=7393fe084b62454e9d96e9d57737a145)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.829541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabec5f-527b-43a9-aea0-5de628fb919a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.829541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 192/377] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/projects/7393fe084b62454e9d96e9d57737a145 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:36.835708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabec5f-527b-43a9-aea0-5de628fb919a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:36.887598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104afb1b-42ce-43e3-804d-7bf89c730ac2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.927021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:36.934653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:36.947051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.952858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.952858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.952858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:36.952858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] RBAC: Adding query filter params (group.id=261c9390f4f3487a8261852bc911f851, role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=9ca640bda24942c38dc244b54e934cf0) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:36.952858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:36.952858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:36.956274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabec5f-527b-43a9-aea0-5de628fb919a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:36.957423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 186/378] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/role_assignments?group.id=261c9390f4f3487a8261852bc911f851&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:36.996414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef510596-763c-47d5-8507-e004380c6500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:36.997213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510596-763c-47d5-8507-e004380c6500 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:36.997601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510596-763c-47d5-8507-e004380c6500 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:36.997979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef510596-763c-47d5-8507-e004380c6500 None admin] PATH_INFO: `/v3/groups/fb0d897ecc6e45ee86e1e47aa3cc8f8d` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.005228 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef510596-763c-47d5-8507-e004380c6500 None admin] RBAC: Authorizing `identity:get_group(group_id=fb0d897ecc6e45ee86e1e47aa3cc8f8d)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.006885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef510596-763c-47d5-8507-e004380c6500 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:37.009125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 193/379] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:36 2022] GET /identity/v3/groups/fb0d897ecc6e45ee86e1e47aa3cc8f8d => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:37.019541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:37.151709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b097ef-c5e2-46a8-8584-7b605e4a2e16 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:37.197426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0c3b1-5518-460c-a194-0de8932ef1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:37.198055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0c3b1-5518-460c-a194-0de8932ef1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.198332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0c3b1-5518-460c-a194-0de8932ef1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.198628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0c3b1-5518-460c-a194-0de8932ef1b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.199361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0c3b1-5518-460c-a194-0de8932ef1b3 None admin] RBAC: Adding query filter params (group.id=fb0d897ecc6e45ee86e1e47aa3cc8f8d, role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=7393fe084b62454e9d96e9d57737a145) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:37.199664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0c3b1-5518-460c-a194-0de8932ef1b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:37.201105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0c3b1-5518-460c-a194-0de8932ef1b3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:37.208222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 187/380] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:37 2022] GET /identity/v3/role_assignments?group.id=fb0d897ecc6e45ee86e1e47aa3cc8f8d&scope.project.id=7393fe084b62454e9d96e9d57737a145&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:37.680699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419881d5-96d3-482c-a94f-aebca5a46732 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.681538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419881d5-96d3-482c-a94f-aebca5a46732 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.681982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419881d5-96d3-482c-a94f-aebca5a46732 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.683700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 194/381] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:37.689191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5f17-2b83-47b6-a789-0fc5fedea0dc None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:37.689959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5f17-2b83-47b6-a789-0fc5fedea0dc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:37.690388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5f17-2b83-47b6-a789-0fc5fedea0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:37.719350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a27d5f17-2b83-47b6-a789-0fc5fedea0dc None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:37.720781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a27d5f17-2b83-47b6-a789-0fc5fedea0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:37.782574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:37.800718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 188/382] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:38.217078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419881d5-96d3-482c-a94f-aebca5a46732 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.348117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419881d5-96d3-482c-a94f-aebca5a46732 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.396146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aae975-891c-49e6-895f-a5b6119ea6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.396936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aae975-891c-49e6-895f-a5b6119ea6fb None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.397277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aae975-891c-49e6-895f-a5b6119ea6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.397556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aae975-891c-49e6-895f-a5b6119ea6fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.415353 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aae975-891c-49e6-895f-a5b6119ea6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-10aae975-891c-49e6-895f-a5b6119ea6fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:38.420334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:38.426856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:38.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 195/383] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:38.428330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d5f17-2b83-47b6-a789-0fc5fedea0dc None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.584888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d5f17-2b83-47b6-a789-0fc5fedea0dc None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.626158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21221172-fc92-45de-9a70-e1f74c7343cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.626820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221172-fc92-45de-9a70-e1f74c7343cf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.627101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221172-fc92-45de-9a70-e1f74c7343cf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.627375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221172-fc92-45de-9a70-e1f74c7343cf None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.628046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21221172-fc92-45de-9a70-e1f74c7343cf None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:38.628349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21221172-fc92-45de-9a70-e1f74c7343cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.630371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21221172-fc92-45de-9a70-e1f74c7343cf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.641768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 189/384] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:38.647464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aae975-891c-49e6-895f-a5b6119ea6fb None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.655426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:38.706863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dd240-b533-4368-a129-4f22810bc81f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.707899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dd240-b533-4368-a129-4f22810bc81f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.708381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dd240-b533-4368-a129-4f22810bc81f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.710221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 190/385] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:02:38.715355 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c84d-bee4-4721-a130-796818397496 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.715855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c84d-bee4-4721-a130-796818397496 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.716336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968c84d-bee4-4721-a130-796818397496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.735034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f968c84d-bee4-4721-a130-796818397496 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:38.736516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f968c84d-bee4-4721-a130-796818397496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:38.810764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:38.860137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aae975-891c-49e6-895f-a5b6119ea6fb None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:38.870780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 191/386] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:38.915962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9551c62-1598-4484-b04f-09acfca4891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:38.916603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551c62-1598-4484-b04f-09acfca4891e None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.916884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551c62-1598-4484-b04f-09acfca4891e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.917153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9551c62-1598-4484-b04f-09acfca4891e None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.918271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9551c62-1598-4484-b04f-09acfca4891e None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:38.919736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9551c62-1598-4484-b04f-09acfca4891e None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:38.921641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 196/387] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:38 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:38.927084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968c84d-bee4-4721-a130-796818397496 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:38.939494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7037ab-0210-41ca-bb87-df6006a4eb70 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.939777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7037ab-0210-41ca-bb87-df6006a4eb70 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.940032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7037ab-0210-41ca-bb87-df6006a4eb70 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.944833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 197/388] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:38.947440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd07336-f78d-4dfe-8c5a-df88fd3c4969 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:38.947718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd07336-f78d-4dfe-8c5a-df88fd3c4969 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:38.947990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd07336-f78d-4dfe-8c5a-df88fd3c4969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:38.978830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7bd07336-f78d-4dfe-8c5a-df88fd3c4969 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:38.980022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7bd07336-f78d-4dfe-8c5a-df88fd3c4969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:39.108260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 198/389] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:39.110495 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968c84d-bee4-4721-a130-796818397496 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.146533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcb307-f077-4cfb-9788-1abf923ac22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.147153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcb307-f077-4cfb-9788-1abf923ac22b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.147425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcb307-f077-4cfb-9788-1abf923ac22b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.147690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcb307-f077-4cfb-9788-1abf923ac22b None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.148852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bcb307-f077-4cfb-9788-1abf923ac22b None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:39.150321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bcb307-f077-4cfb-9788-1abf923ac22b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:39.152258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 192/390] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:38 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:39.157889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd07336-f78d-4dfe-8c5a-df88fd3c4969 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.166706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:39.248551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcb307-f077-4cfb-9788-1abf923ac22b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.284480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd07336-f78d-4dfe-8c5a-df88fd3c4969 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.321203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:39.371190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012d0bc-059a-42a9-ba3d-dcad272574a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.371783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d0bc-059a-42a9-ba3d-dcad272574a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.372072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d0bc-059a-42a9-ba3d-dcad272574a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.372334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d0bc-059a-42a9-ba3d-dcad272574a6 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/groups/261c9390f4f3487a8261852bc911f851/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.374315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012d0bc-059a-42a9-ba3d-dcad272574a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8be5bc6beaa4f6f926f28d045911f2e, group_id=261c9390f4f3487a8261852bc911f851, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:39.375770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012d0bc-059a-42a9-ba3d-dcad272574a6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:39.388545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcb307-f077-4cfb-9788-1abf923ac22b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.431737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac138-8a9c-4500-8a76-d7e8770a4f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.432366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac138-8a9c-4500-8a76-d7e8770a4f2a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.432638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac138-8a9c-4500-8a76-d7e8770a4f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.432905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac138-8a9c-4500-8a76-d7e8770a4f2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.443760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faac138-8a9c-4500-8a76-d7e8770a4f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-1faac138-8a9c-4500-8a76-d7e8770a4f2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:39.449034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:39.449966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:39.450843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 193/391] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:39.451521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751eb0-d03b-45b0-8da8-3ae36f17dc15 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.451521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751eb0-d03b-45b0-8da8-3ae36f17dc15 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.452096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3751eb0-d03b-45b0-8da8-3ae36f17dc15 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.453216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 194/392] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:39.457848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3751eb0-d03b-45b0-8da8-3ae36f17dc15 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.465056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 199/393] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Sep 9 23:02:39 2022] PUT /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/groups/261c9390f4f3487a8261852bc911f851/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:39.466580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed9c8a-a65b-4d73-9dea-a718b63b55fb None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.466937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed9c8a-a65b-4d73-9dea-a718b63b55fb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.467587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed9c8a-a65b-4d73-9dea-a718b63b55fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.491940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-82ed9c8a-a65b-4d73-9dea-a718b63b55fb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:39.493392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-82ed9c8a-a65b-4d73-9dea-a718b63b55fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:39.589288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:39.605783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3751eb0-d03b-45b0-8da8-3ae36f17dc15 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.610439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 200/394] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:39.611866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed9c8a-a65b-4d73-9dea-a718b63b55fb None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.622592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:39.687964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.688567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.688839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.689105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.689819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:39.690184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:39.692220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:39.708450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 195/395] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:39.714253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.757094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed9c8a-a65b-4d73-9dea-a718b63b55fb None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.793101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62bf2b-ecfe-479e-b84c-49d6805044eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.793870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62bf2b-ecfe-479e-b84c-49d6805044eb None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.794266 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62bf2b-ecfe-479e-b84c-49d6805044eb None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.794642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62bf2b-ecfe-479e-b84c-49d6805044eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.806281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62bf2b-ecfe-479e-b84c-49d6805044eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-6d62bf2b-ecfe-479e-b84c-49d6805044eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:39.811510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:39.812590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:39.813579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:39.815522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 201/396] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:39.822365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62bf2b-ecfe-479e-b84c-49d6805044eb None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.854293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac2598-07ee-403f-98bb-5e6efe2b96ae None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:39.905098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0280c-f515-486b-8870-365f83ecb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:39.905791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0280c-f515-486b-8870-365f83ecb685 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:39.906265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:39.906724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0280c-f515-486b-8870-365f83ecb685 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:39.907095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0280c-f515-486b-8870-365f83ecb685 None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:39.908465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0280c-f515-486b-8870-365f83ecb685 None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:39.910250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0280c-f515-486b-8870-365f83ecb685 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:39.912950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 196/397] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:39.918536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0280c-f515-486b-8870-365f83ecb685 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:39.971091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:39.980576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62bf2b-ecfe-479e-b84c-49d6805044eb None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.021435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.022049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.022326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.022641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.023281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:40.023583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.025100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.033548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 202/398] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:40.039748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.041412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0280c-f515-486b-8870-365f83ecb685 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.079498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c84c5-79d1-4085-bc23-562b0f5b9bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.080432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c84c5-79d1-4085-bc23-562b0f5b9bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.080703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c84c5-79d1-4085-bc23-562b0f5b9bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.080968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c84c5-79d1-4085-bc23-562b0f5b9bb8 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.082124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c84c5-79d1-4085-bc23-562b0f5b9bb8 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.084771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c84c5-79d1-4085-bc23-562b0f5b9bb8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.090949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 197/399] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:39 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:40.090949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9d19-ff0d-4e3c-bbb3-ac35669e0c73 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.090949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9d19-ff0d-4e3c-bbb3-ac35669e0c73 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.090949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b9d19-ff0d-4e3c-bbb3-ac35669e0c73 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.091414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 198/400] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:40.092309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9d19-ff0d-4e3c-bbb3-ac35669e0c73 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.154518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcba91-f6ac-45f8-b21f-f35624d68422 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.170288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:40.190868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fda0b-caa3-4051-8533-6eaf2eaddf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.192165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fda0b-caa3-4051-8533-6eaf2eaddf97 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.192446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fda0b-caa3-4051-8533-6eaf2eaddf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.192894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fda0b-caa3-4051-8533-6eaf2eaddf97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.199876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b9d19-ff0d-4e3c-bbb3-ac35669e0c73 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.203560 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:40.204018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fda0b-caa3-4051-8533-6eaf2eaddf97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-280fda0b-caa3-4051-8533-6eaf2eaddf97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:40.209862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:40.217186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 203/401] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:40.217974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fda0b-caa3-4051-8533-6eaf2eaddf97 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.241816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dc99a-8d03-478c-9c4b-40f3b0556938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.242488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc99a-8d03-478c-9c4b-40f3b0556938 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.242771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc99a-8d03-478c-9c4b-40f3b0556938 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.243047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dc99a-8d03-478c-9c4b-40f3b0556938 None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.244605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dc99a-8d03-478c-9c4b-40f3b0556938 None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.246107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dc99a-8d03-478c-9c4b-40f3b0556938 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.247894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 199/402] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:40 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:40.253265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fe02c-8a60-4dad-bc9b-738b56e7011b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.253544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fe02c-8a60-4dad-bc9b-738b56e7011b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.253819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fe02c-8a60-4dad-bc9b-738b56e7011b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.272428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b01fe02c-8a60-4dad-bc9b-738b56e7011b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:40.273616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b01fe02c-8a60-4dad-bc9b-738b56e7011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:40.329099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fda0b-caa3-4051-8533-6eaf2eaddf97 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.342861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 200/403] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:40.343715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fe02c-8a60-4dad-bc9b-738b56e7011b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.382557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b1d6e-e960-4d09-8dcb-da1ab861a100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.383588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b1d6e-e960-4d09-8dcb-da1ab861a100 None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.384075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b1d6e-e960-4d09-8dcb-da1ab861a100 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.384558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b1d6e-e960-4d09-8dcb-da1ab861a100 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0/groups/261c9390f4f3487a8261852bc911f851/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.387827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b1d6e-e960-4d09-8dcb-da1ab861a100 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca640bda24942c38dc244b54e934cf0, group_id=261c9390f4f3487a8261852bc911f851, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.390196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b1d6e-e960-4d09-8dcb-da1ab861a100 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.444465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 204/404] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Sep 9 23:02:40 2022] PUT /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0/groups/261c9390f4f3487a8261852bc911f851/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:40.445847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b1d6e-e960-4d09-8dcb-da1ab861a100 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.461776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fe02c-8a60-4dad-bc9b-738b56e7011b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.463537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:40.493261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:40.532640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.533275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.534128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.534395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.535685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:40.535990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.537749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.559708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 201/405] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:40.566926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.580329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b1d6e-e960-4d09-8dcb-da1ab861a100 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.632409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7da2e-4a75-4d68-bf6f-4d3661d3106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.634054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7da2e-4a75-4d68-bf6f-4d3661d3106b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.634590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7da2e-4a75-4d68-bf6f-4d3661d3106b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.635265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7da2e-4a75-4d68-bf6f-4d3661d3106b None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.637353 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7da2e-4a75-4d68-bf6f-4d3661d3106b None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.639742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7da2e-4a75-4d68-bf6f-4d3661d3106b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.642292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 205/406] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:40 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:40.644254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7da2e-4a75-4d68-bf6f-4d3661d3106b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.723855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddb733-9b94-44e6-8368-935e75a69e5e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.760951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ed93-3648-4aef-a5ad-fdfb6be7a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.761573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ed93-3648-4aef-a5ad-fdfb6be7a070 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.761848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ed93-3648-4aef-a5ad-fdfb6be7a070 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.762116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6ed93-3648-4aef-a5ad-fdfb6be7a070 None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.763291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6ed93-3648-4aef-a5ad-fdfb6be7a070 None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.764817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6ed93-3648-4aef-a5ad-fdfb6be7a070 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.766739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 202/407] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:40 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:40.767554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ed93-3648-4aef-a5ad-fdfb6be7a070 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.783492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7da2e-4a75-4d68-bf6f-4d3661d3106b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.787852 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:40.801975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:40.833380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90d315-bc70-4115-aa6e-21349955ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.834650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90d315-bc70-4115-aa6e-21349955ea3d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.834927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90d315-bc70-4115-aa6e-21349955ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.835208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90d315-bc70-4115-aa6e-21349955ea3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.853844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90d315-bc70-4115-aa6e-21349955ea3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-5f90d315-bc70-4115-aa6e-21349955ea3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:40.859606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:40.860496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:40.861377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 206/408] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:02:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:40.862943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90d315-bc70-4115-aa6e-21349955ea3d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:40.893908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6ed93-3648-4aef-a5ad-fdfb6be7a070 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:40.936094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fa8b9-c4e0-4ecc-babb-a9aef38b9e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:40.936721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fa8b9-c4e0-4ecc-babb-a9aef38b9e55 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:40.936997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fa8b9-c4e0-4ecc-babb-a9aef38b9e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:40.937265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fa8b9-c4e0-4ecc-babb-a9aef38b9e55 None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/groups/261c9390f4f3487a8261852bc911f851/roles/54e2dc31e78b4516b2296086e9eb12c4` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:40.939585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fa8b9-c4e0-4ecc-babb-a9aef38b9e55 None admin] RBAC: Authorizing `identity:create_grant(project_id=e8be5bc6beaa4f6f926f28d045911f2e, group_id=261c9390f4f3487a8261852bc911f851, role_id=54e2dc31e78b4516b2296086e9eb12c4)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:40.941037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fa8b9-c4e0-4ecc-babb-a9aef38b9e55 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:40.981675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90d315-bc70-4115-aa6e-21349955ea3d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:41.011149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 203/409] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Sep 9 23:02:40 2022] PUT /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e/groups/261c9390f4f3487a8261852bc911f851/roles/54e2dc31e78b4516b2296086e9eb12c4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:41.013188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fa8b9-c4e0-4ecc-babb-a9aef38b9e55 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:41.021812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a8ae6-f260-4279-ab38-e6cea80e4ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:41.022575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a8ae6-f260-4279-ab38-e6cea80e4ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.022856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a8ae6-f260-4279-ab38-e6cea80e4ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.023127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a8ae6-f260-4279-ab38-e6cea80e4ab8 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.024434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a8ae6-f260-4279-ab38-e6cea80e4ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:41.030822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a8ae6-f260-4279-ab38-e6cea80e4ab8 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:41.030822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 207/410] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:40 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:41.037387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a8ae6-f260-4279-ab38-e6cea80e4ab8 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:41.108007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:41.209846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:41.221413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fa8b9-c4e0-4ecc-babb-a9aef38b9e55 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:41.241491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a8ae6-f260-4279-ab38-e6cea80e4ab8 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:41.261551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:41.262156 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.262705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.262974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.263633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:41.264012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:41.266320 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:41.276391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 204/411] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:41.283897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:41.291395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fd138-b7d7-4904-b0f4-46dae6b725a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:41.292059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fd138-b7d7-4904-b0f4-46dae6b725a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.292332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fd138-b7d7-4904-b0f4-46dae6b725a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.292601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fd138-b7d7-4904-b0f4-46dae6b725a4 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0/groups/261c9390f4f3487a8261852bc911f851/roles/54e2dc31e78b4516b2296086e9eb12c4` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.294796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fd138-b7d7-4904-b0f4-46dae6b725a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ca640bda24942c38dc244b54e934cf0, group_id=261c9390f4f3487a8261852bc911f851, role_id=54e2dc31e78b4516b2296086e9eb12c4)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:41.296690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fd138-b7d7-4904-b0f4-46dae6b725a4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:41.382320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 208/412] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Sep 9 23:02:41 2022] PUT /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0/groups/261c9390f4f3487a8261852bc911f851/roles/54e2dc31e78b4516b2296086e9eb12c4 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:41.414705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7540b69-8415-47bb-bd4d-59311142d52a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:41.481905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472cfe1-d7d8-4e04-bf18-4bdccea48a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:41.482648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cfe1-d7d8-4e04-bf18-4bdccea48a4a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.482927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cfe1-d7d8-4e04-bf18-4bdccea48a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.483195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cfe1-d7d8-4e04-bf18-4bdccea48a4a None admin] PATH_INFO: `/v3/groups/fb0d897ecc6e45ee86e1e47aa3cc8f8d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.486187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472cfe1-d7d8-4e04-bf18-4bdccea48a4a None admin] RBAC: Authorizing `identity:get_group(group_id=fb0d897ecc6e45ee86e1e47aa3cc8f8d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:41.487863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472cfe1-d7d8-4e04-bf18-4bdccea48a4a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:41.490696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 205/413] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:41 2022] GET /identity/v3/groups/fb0d897ecc6e45ee86e1e47aa3cc8f8d => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:41.496635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fd138-b7d7-4904-b0f4-46dae6b725a4 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:41.622856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:41.633122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fd138-b7d7-4904-b0f4-46dae6b725a4 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:41.668835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a96cc-ab3e-4b52-8c4d-29bca23c7f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:41.669575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a96cc-ab3e-4b52-8c4d-29bca23c7f3a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.669836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a96cc-ab3e-4b52-8c4d-29bca23c7f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.670104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a96cc-ab3e-4b52-8c4d-29bca23c7f3a None admin] PATH_INFO: `/v3/projects/7393fe084b62454e9d96e9d57737a145` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.671253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a96cc-ab3e-4b52-8c4d-29bca23c7f3a None admin] RBAC: Authorizing `identity:get_project(project_id=7393fe084b62454e9d96e9d57737a145)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:41.672667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a96cc-ab3e-4b52-8c4d-29bca23c7f3a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:41.674734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 209/414] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:41 2022] GET /identity/v3/projects/7393fe084b62454e9d96e9d57737a145 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:41.680542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472cfe1-d7d8-4e04-bf18-4bdccea48a4a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:41.712010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:41.811356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472cfe1-d7d8-4e04-bf18-4bdccea48a4a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:41.854145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423168c5-6b02-4225-855d-4f7b4be433f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:41.854851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423168c5-6b02-4225-855d-4f7b4be433f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.855127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423168c5-6b02-4225-855d-4f7b4be433f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.855398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423168c5-6b02-4225-855d-4f7b4be433f0 None admin] PATH_INFO: `/v3/projects/7393fe084b62454e9d96e9d57737a145/groups/fb0d897ecc6e45ee86e1e47aa3cc8f8d/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.857325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423168c5-6b02-4225-855d-4f7b4be433f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7393fe084b62454e9d96e9d57737a145, group_id=fb0d897ecc6e45ee86e1e47aa3cc8f8d, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:41.858783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423168c5-6b02-4225-855d-4f7b4be433f0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:41.928839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 206/415] 10.222.0.20 () {58 vars in 1442 bytes} [Fri Sep 9 23:02:41 2022] PUT /identity/v3/projects/7393fe084b62454e9d96e9d57737a145/groups/fb0d897ecc6e45ee86e1e47aa3cc8f8d/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:02:41.981239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e43065-37ec-4577-a15c-5dc4716eb73f None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.981546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e43065-37ec-4577-a15c-5dc4716eb73f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.981740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e43065-37ec-4577-a15c-5dc4716eb73f None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:41.983362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 210/416] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:41.990458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633bdf4-af49-43d4-a671-ba234aa38e8e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:41.990785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633bdf4-af49-43d4-a671-ba234aa38e8e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:41.991003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633bdf4-af49-43d4-a671-ba234aa38e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:42.015429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3633bdf4-af49-43d4-a671-ba234aa38e8e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:42.016625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3633bdf4-af49-43d4-a671-ba234aa38e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:42.105398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 207/417] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:42.430010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e43065-37ec-4577-a15c-5dc4716eb73f None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:42.571026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e43065-37ec-4577-a15c-5dc4716eb73f None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:42.618414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92310a29-509a-4cd4-8fe1-bea1bade9135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:42.619046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92310a29-509a-4cd4-8fe1-bea1bade9135 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:42.619340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92310a29-509a-4cd4-8fe1-bea1bade9135 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:42.619590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92310a29-509a-4cd4-8fe1-bea1bade9135 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:42.637152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92310a29-509a-4cd4-8fe1-bea1bade9135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-92310a29-509a-4cd4-8fe1-bea1bade9135 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:42.641995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:42.642933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:42.643718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 211/418] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:42.649076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633bdf4-af49-43d4-a671-ba234aa38e8e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:42.668271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:42.762326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633bdf4-af49-43d4-a671-ba234aa38e8e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:42.827047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c1aa8-f61a-4d6f-94c1-415e5d6cc7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:42.827847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c1aa8-f61a-4d6f-94c1-415e5d6cc7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:42.828008 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c1aa8-f61a-4d6f-94c1-415e5d6cc7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:42.828163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c1aa8-f61a-4d6f-94c1-415e5d6cc7d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:42.828822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c1aa8-f61a-4d6f-94c1-415e5d6cc7d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:42.829121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c1aa8-f61a-4d6f-94c1-415e5d6cc7d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:42.830856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c1aa8-f61a-4d6f-94c1-415e5d6cc7d4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:42.840353 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 208/419] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:42.845996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92310a29-509a-4cd4-8fe1-bea1bade9135 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:42.878115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:42.990819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92310a29-509a-4cd4-8fe1-bea1bade9135 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:43.024616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df421ea-c7ef-4361-8530-95802debf939 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.024801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df421ea-c7ef-4361-8530-95802debf939 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.025072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df421ea-c7ef-4361-8530-95802debf939 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.026512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 209/420] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:43.037201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1560b-d9a1-46c2-9f3b-d02ce9e2b5bf None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.037488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1560b-d9a1-46c2-9f3b-d02ce9e2b5bf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.037757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1560b-d9a1-46c2-9f3b-d02ce9e2b5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.041741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e361b-7349-4c08-8591-3147ca40d77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:43.042362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e361b-7349-4c08-8591-3147ca40d77b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.042656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e361b-7349-4c08-8591-3147ca40d77b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.042933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e361b-7349-4c08-8591-3147ca40d77b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.079339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f9e1560b-d9a1-46c2-9f3b-d02ce9e2b5bf None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:43.080839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f9e1560b-d9a1-46c2-9f3b-d02ce9e2b5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:43.139881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e361b-7349-4c08-8591-3147ca40d77b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:43.142224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e361b-7349-4c08-8591-3147ca40d77b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:43.178367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:43.221952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 210/421] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:43.301988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 212/422] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:43.310365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1560b-d9a1-46c2-9f3b-d02ce9e2b5bf None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:43.310596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:43.462580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1560b-d9a1-46c2-9f3b-d02ce9e2b5bf None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:43.501766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452e4c-9f33-4040-be68-82280a43365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:43.502415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452e4c-9f33-4040-be68-82280a43365d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.502692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452e4c-9f33-4040-be68-82280a43365d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.502958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452e4c-9f33-4040-be68-82280a43365d None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.504660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7452e4c-9f33-4040-be68-82280a43365d None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:43.506112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7452e4c-9f33-4040-be68-82280a43365d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:43.508102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 211/423] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:43 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:43.513622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e361b-7349-4c08-8591-3147ca40d77b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:43.666265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e361b-7349-4c08-8591-3147ca40d77b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:43.683621 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452e4c-9f33-4040-be68-82280a43365d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:43.691782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:43.722147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434048c-efbd-4114-aa98-812d1a9da011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:43.725009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434048c-efbd-4114-aa98-812d1a9da011 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.725509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434048c-efbd-4114-aa98-812d1a9da011 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.725975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434048c-efbd-4114-aa98-812d1a9da011 None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.728188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434048c-efbd-4114-aa98-812d1a9da011 None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:43.730484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434048c-efbd-4114-aa98-812d1a9da011 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:43.734585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 213/424] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:43 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:43.740776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434048c-efbd-4114-aa98-812d1a9da011 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:43.812456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:43.865106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452e4c-9f33-4040-be68-82280a43365d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:43.900857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434048c-efbd-4114-aa98-812d1a9da011 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:43.933192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586da2-c6ca-41bb-9f6a-0663346b011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:43.934203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586da2-c6ca-41bb-9f6a-0663346b011d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.934705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586da2-c6ca-41bb-9f6a-0663346b011d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.935192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586da2-c6ca-41bb-9f6a-0663346b011d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.948031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae586da2-c6ca-41bb-9f6a-0663346b011d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:43.949474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7fe66-d2aa-41bd-87bf-d5558f83f259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:43.950079 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7fe66-d2aa-41bd-87bf-d5558f83f259 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.950373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7fe66-d2aa-41bd-87bf-d5558f83f259 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.952424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7fe66-d2aa-41bd-87bf-d5558f83f259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.953151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7fe66-d2aa-41bd-87bf-d5558f83f259 None admin] RBAC: Adding query filter params (group.id=261c9390f4f3487a8261852bc911f851, role.id=453242bdc829422490366038c62c149a, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:43.953459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7fe66-d2aa-41bd-87bf-d5558f83f259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:43.955175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7fe66-d2aa-41bd-87bf-d5558f83f259 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-ae586da2-c6ca-41bb-9f6a-0663346b011d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:43.955896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:43.956770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 09 23:02:43.958082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:43.961079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 212/425] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:43.967592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01479907-0710-4ba6-b7a2-0d321b6bac6d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.967905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01479907-0710-4ba6-b7a2-0d321b6bac6d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.968152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01479907-0710-4ba6-b7a2-0d321b6bac6d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:43.969769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 213/426] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:43.970681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 214/427] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:43 2022] GET /identity/v3/role_assignments?group.id=261c9390f4f3487a8261852bc911f851&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=453242bdc829422490366038c62c149a => generated 611 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:43.970794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01479907-0710-4ba6-b7a2-0d321b6bac6d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:43.989099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf63d34-2fb4-4bd0-a7dc-43ee58620ce2 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:43.989099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf63d34-2fb4-4bd0-a7dc-43ee58620ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:43.989381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf63d34-2fb4-4bd0-a7dc-43ee58620ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.020463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-bbf63d34-2fb4-4bd0-a7dc-43ee58620ce2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:44.021958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bbf63d34-2fb4-4bd0-a7dc-43ee58620ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:44.118735 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:44.171626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01479907-0710-4ba6-b7a2-0d321b6bac6d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:44.183109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 215/428] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:44.203841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:44.209711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da5a81-6a46-4d2b-af48-27bad3587152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:44.216032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da5a81-6a46-4d2b-af48-27bad3587152 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.216032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da5a81-6a46-4d2b-af48-27bad3587152 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.216032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da5a81-6a46-4d2b-af48-27bad3587152 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.216032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da5a81-6a46-4d2b-af48-27bad3587152 None admin] RBAC: Adding query filter params (name=member) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:44.216032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da5a81-6a46-4d2b-af48-27bad3587152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:44.216032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da5a81-6a46-4d2b-af48-27bad3587152 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:44.222546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 214/429] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:02:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:44.228832 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf63d34-2fb4-4bd0-a7dc-43ee58620ce2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:44.243932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7399-655f-44be-85d6-8c4c7364ba0d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.243932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7399-655f-44be-85d6-8c4c7364ba0d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.243932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a7399-655f-44be-85d6-8c4c7364ba0d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.243932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 215/430] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:02:44.249211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa7232-23c3-4096-bdd3-a38c32ad15da None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.249540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa7232-23c3-4096-bdd3-a38c32ad15da None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.249817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa7232-23c3-4096-bdd3-a38c32ad15da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.270518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c1aa7232-23c3-4096-bdd3-a38c32ad15da None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:44.272070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c1aa7232-23c3-4096-bdd3-a38c32ad15da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:44.368508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 216/431] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:44.381110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf63d34-2fb4-4bd0-a7dc-43ee58620ce2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:44.418627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fbc7a-adfa-48a9-b759-c37f619afe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:44.419366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fbc7a-adfa-48a9-b759-c37f619afe23 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.419717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fbc7a-adfa-48a9-b759-c37f619afe23 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.420066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fbc7a-adfa-48a9-b759-c37f619afe23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.475392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:44.495742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fbc7a-adfa-48a9-b759-c37f619afe23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:44.497217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fbc7a-adfa-48a9-b759-c37f619afe23 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:44.508783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa7232-23c3-4096-bdd3-a38c32ad15da None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:44.624919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 216/432] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:44.631367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fbc7a-adfa-48a9-b759-c37f619afe23 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:44.641508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa7232-23c3-4096-bdd3-a38c32ad15da None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:44.685706 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:44.697310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83a5f-eb95-4bc2-af40-5dcd92d0d073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:44.698220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83a5f-eb95-4bc2-af40-5dcd92d0d073 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.698530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83a5f-eb95-4bc2-af40-5dcd92d0d073 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.698827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd83a5f-eb95-4bc2-af40-5dcd92d0d073 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.715474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd83a5f-eb95-4bc2-af40-5dcd92d0d073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-ebd83a5f-eb95-4bc2-af40-5dcd92d0d073 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:44.720889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:44.721855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:44.722664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:44.728776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 217/433] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:44.728776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83a5f-eb95-4bc2-af40-5dcd92d0d073 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:44.742828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:44.770750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fbc7a-adfa-48a9-b759-c37f619afe23 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:44.822541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d02a31-44d9-4604-aea6-4da73422ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:44.823275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d02a31-44d9-4604-aea6-4da73422ff9b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.823623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d02a31-44d9-4604-aea6-4da73422ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.823971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d02a31-44d9-4604-aea6-4da73422ff9b None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.825648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d02a31-44d9-4604-aea6-4da73422ff9b None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:44.827502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d02a31-44d9-4604-aea6-4da73422ff9b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:44.829281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 217/434] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:44 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:44.830691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d02a31-44d9-4604-aea6-4da73422ff9b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:44.853598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd83a5f-eb95-4bc2-af40-5dcd92d0d073 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:44.887488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561310b-217c-468c-a23b-489c72454c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:44.888115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561310b-217c-468c-a23b-489c72454c55 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.888385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561310b-217c-468c-a23b-489c72454c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.888648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561310b-217c-468c-a23b-489c72454c55 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.906279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561310b-217c-468c-a23b-489c72454c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0561310b-217c-468c-a23b-489c72454c55 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:44.911622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:44.912526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:44.913327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:44.917639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 218/435] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:44.920446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c2fe4-0fda-4d40-9924-7691452db35f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.920729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c2fe4-0fda-4d40-9924-7691452db35f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.921004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c2fe4-0fda-4d40-9924-7691452db35f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.922856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 219/436] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:44.923560 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c2fe4-0fda-4d40-9924-7691452db35f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:44.942113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d02a31-44d9-4604-aea6-4da73422ff9b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:44.970687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:44.977806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:44.978435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:44.978711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:44.979028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:44.979621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:44.979916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:44.981259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:44.989898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 218/437] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:44.990757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.007541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.025858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c2fe4-0fda-4d40-9924-7691452db35f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.058316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9598ec4-da5f-41ce-9571-68163dfb1c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.058958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598ec4-da5f-41ce-9571-68163dfb1c1a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.059229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598ec4-da5f-41ce-9571-68163dfb1c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.059492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9598ec4-da5f-41ce-9571-68163dfb1c1a None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.060579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598ec4-da5f-41ce-9571-68163dfb1c1a None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.061999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9598ec4-da5f-41ce-9571-68163dfb1c1a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.063574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 220/438] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:44 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:45.064677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241ae4c-c25f-42bd-823c-19a397deef42 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.064919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241ae4c-c25f-42bd-823c-19a397deef42 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.065181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241ae4c-c25f-42bd-823c-19a397deef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.083230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7241ae4c-c25f-42bd-823c-19a397deef42 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:45.084411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7241ae4c-c25f-42bd-823c-19a397deef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:45.095669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ffa8a-1016-43a7-afa0-1f3de5085722 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.128465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.129120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.129394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.129923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.130311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:45.130653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.131991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.139900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 219/439] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:45.140488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.152758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 221/440] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:45.153589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241ae4c-c25f-42bd-823c-19a397deef42 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.207942 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.221130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.251011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fce450-0e04-4ac2-9398-a27e950a40bf None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.263569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241ae4c-c25f-42bd-823c-19a397deef42 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.288720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f37edb-6236-44ec-8925-6cceb516925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.289326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f37edb-6236-44ec-8925-6cceb516925e None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.289606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f37edb-6236-44ec-8925-6cceb516925e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.289878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f37edb-6236-44ec-8925-6cceb516925e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.299153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.300152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.300428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.300697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.302374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] RBAC: Adding query filter params (group.id=261c9390f4f3487a8261852bc911f851, role.id=453242bdc829422490366038c62c149a, scope.project.id=9ca640bda24942c38dc244b54e934cf0) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:45.302719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.304460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.322380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 222/441] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/role_assignments?group.id=261c9390f4f3487a8261852bc911f851&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=453242bdc829422490366038c62c149a => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:45.322380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.389441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f37edb-6236-44ec-8925-6cceb516925e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.391361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f37edb-6236-44ec-8925-6cceb516925e None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.456890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d3778-238c-43be-b01e-5bcae92259ed None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.465925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.485043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.500206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440708a-743c-4bd6-91d7-6edc7352ec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.500816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440708a-743c-4bd6-91d7-6edc7352ec30 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.501147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440708a-743c-4bd6-91d7-6edc7352ec30 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.501367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440708a-743c-4bd6-91d7-6edc7352ec30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.519110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 220/442] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:45.521867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f37edb-6236-44ec-8925-6cceb516925e None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.584126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440708a-743c-4bd6-91d7-6edc7352ec30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.585611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440708a-743c-4bd6-91d7-6edc7352ec30 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.628754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f37edb-6236-44ec-8925-6cceb516925e None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.663193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e45f5-6dcb-4ef9-8357-717834d4117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.663813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e45f5-6dcb-4ef9-8357-717834d4117d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.664089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e45f5-6dcb-4ef9-8357-717834d4117d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.664359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e45f5-6dcb-4ef9-8357-717834d4117d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.674092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e45f5-6dcb-4ef9-8357-717834d4117d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-831e45f5-6dcb-4ef9-8357-717834d4117d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:45.679409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:45.680325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:45.683231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 221/443] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:45.683892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 223/444] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:45.683972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e45f5-6dcb-4ef9-8357-717834d4117d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.686474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440708a-743c-4bd6-91d7-6edc7352ec30 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.695502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.714964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.783987 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e45f5-6dcb-4ef9-8357-717834d4117d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.785939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440708a-743c-4bd6-91d7-6edc7352ec30 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.818208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23f77c-e92b-48ae-8e48-f64c807df12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.818846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f77c-e92b-48ae-8e48-f64c807df12d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.819120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f77c-e92b-48ae-8e48-f64c807df12d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.819286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9a482-11a2-4954-87f5-d13d2ba5d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.819408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23f77c-e92b-48ae-8e48-f64c807df12d None admin] PATH_INFO: `/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.819909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9a482-11a2-4954-87f5-d13d2ba5d726 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.820189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9a482-11a2-4954-87f5-d13d2ba5d726 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.820461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9a482-11a2-4954-87f5-d13d2ba5d726 None admin] PATH_INFO: `/v3/projects/9ca640bda24942c38dc244b54e934cf0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.820548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23f77c-e92b-48ae-8e48-f64c807df12d None admin] RBAC: Authorizing `identity:get_project(project_id=e8be5bc6beaa4f6f926f28d045911f2e)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.821572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9a482-11a2-4954-87f5-d13d2ba5d726 None admin] RBAC: Authorizing `identity:get_project(project_id=9ca640bda24942c38dc244b54e934cf0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.821874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23f77c-e92b-48ae-8e48-f64c807df12d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.823026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9a482-11a2-4954-87f5-d13d2ba5d726 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.823566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 222/445] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/projects/e8be5bc6beaa4f6f926f28d045911f2e => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:45.824311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23f77c-e92b-48ae-8e48-f64c807df12d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.825025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 224/446] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/projects/9ca640bda24942c38dc244b54e934cf0 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:45.829088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9a482-11a2-4954-87f5-d13d2ba5d726 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.903860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.924646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:45.928030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23f77c-e92b-48ae-8e48-f64c807df12d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.928161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9a482-11a2-4954-87f5-d13d2ba5d726 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:45.962683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1192b80-0312-4f4e-a32b-0029f233819d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.963125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:45.963299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192b80-0312-4f4e-a32b-0029f233819d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.963572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192b80-0312-4f4e-a32b-0029f233819d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.963722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:45.963842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192b80-0312-4f4e-a32b-0029f233819d None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.964000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:45.964271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:45.964924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:45.965020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1192b80-0312-4f4e-a32b-0029f233819d None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.965222 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:45.966386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1192b80-0312-4f4e-a32b-0029f233819d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.966597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:45.968448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 225/447] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:45.969190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1192b80-0312-4f4e-a32b-0029f233819d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:45.981261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 223/448] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:45.982075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:46.070864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1192b80-0312-4f4e-a32b-0029f233819d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.079684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9e0df-6986-432c-98c3-1d41f7d62e5b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.106646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab93a5a-6c8e-4de0-8fd9-2d91b39952d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.107638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab93a5a-6c8e-4de0-8fd9-2d91b39952d4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.108095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab93a5a-6c8e-4de0-8fd9-2d91b39952d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.108550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab93a5a-6c8e-4de0-8fd9-2d91b39952d4 None admin] PATH_INFO: `/v3/groups/261c9390f4f3487a8261852bc911f851` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.110333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab93a5a-6c8e-4de0-8fd9-2d91b39952d4 None admin] RBAC: Authorizing `identity:get_group(group_id=261c9390f4f3487a8261852bc911f851)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.112304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.112896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.112999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab93a5a-6c8e-4de0-8fd9-2d91b39952d4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.113167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.114017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.114986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] RBAC: Adding query filter params (group.id=261c9390f4f3487a8261852bc911f851, role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:46.115285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.115594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 226/449] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/groups/261c9390f4f3487a8261852bc911f851 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:46.116362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab93a5a-6c8e-4de0-8fd9-2d91b39952d4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:46.116693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.136684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 224/450] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:45 2022] GET /identity/v3/role_assignments?group.id=261c9390f4f3487a8261852bc911f851&scope.project.id=e8be5bc6beaa4f6f926f28d045911f2e&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:46.136684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:46.137014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:46.158538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:46.230827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab93a5a-6c8e-4de0-8fd9-2d91b39952d4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.245236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8edf51-7053-4343-8754-e4aa19d58bc2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.272620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43aa13-a833-4d35-84e0-ca67fab6c99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.273242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43aa13-a833-4d35-84e0-ca67fab6c99a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.273521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43aa13-a833-4d35-84e0-ca67fab6c99a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.273793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43aa13-a833-4d35-84e0-ca67fab6c99a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.284962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.286786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.287164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.287847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.288675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] RBAC: Adding query filter params (group.id=261c9390f4f3487a8261852bc911f851, role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=9ca640bda24942c38dc244b54e934cf0) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:46.291059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.291059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.302377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 225/451] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:46 2022] GET /identity/v3/role_assignments?group.id=261c9390f4f3487a8261852bc911f851&scope.project.id=9ca640bda24942c38dc244b54e934cf0&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:46.355610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43aa13-a833-4d35-84e0-ca67fab6c99a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.357021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43aa13-a833-4d35-84e0-ca67fab6c99a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.371348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:46.472662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 227/452] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:46.479893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:46.581263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:46.591117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4e8de-cec3-482e-804a-2d4ac2eb182d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.625622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7a9d6-719d-4880-81c7-d3f8ca349acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.626174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7a9d6-719d-4880-81c7-d3f8ca349acb None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.626480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7a9d6-719d-4880-81c7-d3f8ca349acb None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.626750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7a9d6-719d-4880-81c7-d3f8ca349acb None admin] PATH_INFO: `/v3/projects/7393fe084b62454e9d96e9d57737a145` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.627873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7a9d6-719d-4880-81c7-d3f8ca349acb None admin] RBAC: Authorizing `identity:get_project(project_id=7393fe084b62454e9d96e9d57737a145)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.629432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7a9d6-719d-4880-81c7-d3f8ca349acb None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.631114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 226/453] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:02:46 2022] GET /identity/v3/projects/7393fe084b62454e9d96e9d57737a145 => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:46.636704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43aa13-a833-4d35-84e0-ca67fab6c99a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:46.743826 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43aa13-a833-4d35-84e0-ca67fab6c99a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.754937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:46.778384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b6ae6-6100-4c30-87b3-e803d55e6a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.779015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6ae6-6100-4c30-87b3-e803d55e6a3e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.779292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6ae6-6100-4c30-87b3-e803d55e6a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.779562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b6ae6-6100-4c30-87b3-e803d55e6a3e None admin] PATH_INFO: `/v3/groups/fb0d897ecc6e45ee86e1e47aa3cc8f8d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.780674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b6ae6-6100-4c30-87b3-e803d55e6a3e None admin] RBAC: Authorizing `identity:get_group(group_id=fb0d897ecc6e45ee86e1e47aa3cc8f8d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.782117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b6ae6-6100-4c30-87b3-e803d55e6a3e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.783729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 228/454] 10.222.0.20 () {56 vars in 1180 bytes} [Fri Sep 9 23:02:46 2022] GET /identity/v3/groups/fb0d897ecc6e45ee86e1e47aa3cc8f8d => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:46.790271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7a9d6-719d-4880-81c7-d3f8ca349acb None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:46.890075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7a9d6-719d-4880-81c7-d3f8ca349acb None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:46.923292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c653c81-083c-4350-85a7-75c4d59de2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:46.923897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c653c81-083c-4350-85a7-75c4d59de2cf None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:46.924177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c653c81-083c-4350-85a7-75c4d59de2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:46.924443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c653c81-083c-4350-85a7-75c4d59de2cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:46.925157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c653c81-083c-4350-85a7-75c4d59de2cf None admin] RBAC: Adding query filter params (group.id=fb0d897ecc6e45ee86e1e47aa3cc8f8d, role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=7393fe084b62454e9d96e9d57737a145) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:46.925462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c653c81-083c-4350-85a7-75c4d59de2cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:46.927570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c653c81-083c-4350-85a7-75c4d59de2cf None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:46.933169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 227/455] 10.222.0.20 () {56 vars in 1376 bytes} [Fri Sep 9 23:02:46 2022] GET /identity/v3/role_assignments?group.id=fb0d897ecc6e45ee86e1e47aa3cc8f8d&scope.project.id=7393fe084b62454e9d96e9d57737a145&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:49.780088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec815af-970d-4c82-b3bb-be13984ea24e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:49.781064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec815af-970d-4c82-b3bb-be13984ea24e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:49.781540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec815af-970d-4c82-b3bb-be13984ea24e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:49.783325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 229/456] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:49.789641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b007b-0acf-4cb8-82a0-96ce134ae226 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:49.790548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b007b-0acf-4cb8-82a0-96ce134ae226 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:49.791017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b007b-0acf-4cb8-82a0-96ce134ae226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:49.817604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-927b007b-0acf-4cb8-82a0-96ce134ae226 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:49.819159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-927b007b-0acf-4cb8-82a0-96ce134ae226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:49.840323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:49.913039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 228/457] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:50.119135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee420189-7144-4611-9793-e4a64141ed6d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.119804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee420189-7144-4611-9793-e4a64141ed6d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.120279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee420189-7144-4611-9793-e4a64141ed6d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.122032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 230/458] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:50.127607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae6602-0d72-46c9-b760-73bd9a41010b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.127901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae6602-0d72-46c9-b760-73bd9a41010b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.128180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae6602-0d72-46c9-b760-73bd9a41010b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.150541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-41ae6602-0d72-46c9-b760-73bd9a41010b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:50.151740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-41ae6602-0d72-46c9-b760-73bd9a41010b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:50.248533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 229/459] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:50.272566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cd47-9384-4e9e-83bf-09bc78891b86 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.273145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cd47-9384-4e9e-83bf-09bc78891b86 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.273606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2cd47-9384-4e9e-83bf-09bc78891b86 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.275391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 231/460] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:50.280384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2895b2-60ce-4b56-aa52-01135090dfff None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.280673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2895b2-60ce-4b56-aa52-01135090dfff None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.281039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2895b2-60ce-4b56-aa52-01135090dfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.306796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6f2895b2-60ce-4b56-aa52-01135090dfff None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:50.307986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6f2895b2-60ce-4b56-aa52-01135090dfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:50.336260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2cd47-9384-4e9e-83bf-09bc78891b86 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:50.415828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 230/461] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:50.487988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2cd47-9384-4e9e-83bf-09bc78891b86 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:50.503608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c460dc-4e96-45a5-80c7-58819aa4c1d2 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.504292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c460dc-4e96-45a5-80c7-58819aa4c1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.507671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c460dc-4e96-45a5-80c7-58819aa4c1d2 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.507671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 231/462] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:02:50.515660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca828d4-449e-43a3-ad72-2175425891ac None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.516183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca828d4-449e-43a3-ad72-2175425891ac None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.516657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca828d4-449e-43a3-ad72-2175425891ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.534557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:50.535207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.536048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.536315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.551190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-eca828d4-449e-43a3-ad72-2175425891ac None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:50.552382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-eca828d4-449e-43a3-ad72-2175425891ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:50.553696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:50.555163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:50.564957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:50.586407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:50.588377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:50.589225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:50.589225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:50.589225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:50.589225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 232/463] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:02:50.595701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:50.647020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:50.668624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 232/464] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:02:50.669667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca828d4-449e-43a3-ad72-2175425891ac None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:50.789809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca828d4-449e-43a3-ad72-2175425891ac None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:50.797170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc8ed5-84b4-402e-8bd5-82ad84f2b60e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:50.826231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:50.827113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.827597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.828032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.839895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:50.841578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:50.843826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:50.846205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:50.846907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:50.847254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:50.847596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:50.848519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:50.848877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:50.850268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:50.860514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:50.861443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:50.862270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:50.862270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:50.862270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:50.864330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 233/465] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:50.865550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:50.871875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 233/466] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:50.872809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:50.973295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:50.982947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54629b09-6698-4189-a5e7-83a907133771 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:50.984686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b04ad-adcb-4abb-ad5a-476a23b3bdd0 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.017808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.018430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.018697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d165da-8014-4b9b-b381-7e36306b52c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.018827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.018957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.019315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d165da-8014-4b9b-b381-7e36306b52c7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.019627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:51.019722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d165da-8014-4b9b-b381-7e36306b52c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.019907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d165da-8014-4b9b-b381-7e36306b52c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.020004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.021362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.030638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d165da-8014-4b9b-b381-7e36306b52c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.032102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d165da-8014-4b9b-b381-7e36306b52c7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.032369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 234/467] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:51.037367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-74d165da-8014-4b9b-b381-7e36306b52c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:51.047038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:51.048133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:51.049181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:51.049181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:51.049181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:51.049181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 234/468] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:51.049880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625858-007d-4a49-b10c-40da2e12a19c None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.050158 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625858-007d-4a49-b10c-40da2e12a19c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.050472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625858-007d-4a49-b10c-40da2e12a19c None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.054466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 235/469] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:51.054466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625858-007d-4a49-b10c-40da2e12a19c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.104007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:51.139533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a385e-ad5b-4a5b-9660-bd7ec576d94a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.158202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625858-007d-4a49-b10c-40da2e12a19c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.172090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d08188-d406-4096-a405-0e82baa6e5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.172682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d08188-d406-4096-a405-0e82baa6e5d6 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.172955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d08188-d406-4096-a405-0e82baa6e5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.173219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d08188-d406-4096-a405-0e82baa6e5d6 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.173852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d08188-d406-4096-a405-0e82baa6e5d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.175297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d08188-d406-4096-a405-0e82baa6e5d6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.193809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.196344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.196344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.196344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.204734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.206432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:51.220454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:51.227755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:51.228629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:51.228629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:51.228629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:51.228629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 236/470] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:51.228629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.277032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 235/471] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:51 2022] POST /identity/v3/users => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:51.278107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d08188-d406-4096-a405-0e82baa6e5d6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.287847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:51.325393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:51.331300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e357da-7fee-40cf-aebe-2ab3a5226b83 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.374317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973687a-ff7b-401a-aaa1-26cb1ab76aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.375261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973687a-ff7b-401a-aaa1-26cb1ab76aaf None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.375590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973687a-ff7b-401a-aaa1-26cb1ab76aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.375861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973687a-ff7b-401a-aaa1-26cb1ab76aaf None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.376522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973687a-ff7b-401a-aaa1-26cb1ab76aaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.378025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973687a-ff7b-401a-aaa1-26cb1ab76aaf None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.389332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d08188-d406-4096-a405-0e82baa6e5d6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.430484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d65aa5-64e4-4d6f-b565-a77e11c5681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.431120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d65aa5-64e4-4d6f-b565-a77e11c5681c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.431390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d65aa5-64e4-4d6f-b565-a77e11c5681c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.431661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d65aa5-64e4-4d6f-b565-a77e11c5681c None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.432323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d65aa5-64e4-4d6f-b565-a77e11c5681c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:51.432625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d65aa5-64e4-4d6f-b565-a77e11c5681c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.434038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d65aa5-64e4-4d6f-b565-a77e11c5681c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.447054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 236/472] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:51.450606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61657f25-8b21-4ff9-9091-d04465bd6dcd None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.450890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61657f25-8b21-4ff9-9091-d04465bd6dcd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.451168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61657f25-8b21-4ff9-9091-d04465bd6dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.461258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 237/473] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:51 2022] POST /identity/v3/users => generated 248 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:51.462675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973687a-ff7b-401a-aaa1-26cb1ab76aaf None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.475613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-61657f25-8b21-4ff9-9091-d04465bd6dcd None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:51.478136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-61657f25-8b21-4ff9-9091-d04465bd6dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:51.562661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:51.571861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 237/474] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:51.572637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61657f25-8b21-4ff9-9091-d04465bd6dcd None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.629715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973687a-ff7b-401a-aaa1-26cb1ab76aaf None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.699388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.700059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.700273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.700546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.701188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:51.707899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.707899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.719593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 238/475] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:51.719759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61657f25-8b21-4ff9-9091-d04465bd6dcd None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.724457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.759820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:51.767769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220328aa-6525-428c-9154-c65ef0b196fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.768617 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220328aa-6525-428c-9154-c65ef0b196fb None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.769224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220328aa-6525-428c-9154-c65ef0b196fb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.770209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220328aa-6525-428c-9154-c65ef0b196fb None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.771573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220328aa-6525-428c-9154-c65ef0b196fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.773197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220328aa-6525-428c-9154-c65ef0b196fb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.854022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0a1a3-0ee0-499f-ad97-d2494c5e985b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:51.894380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 238/476] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:51 2022] POST /identity/v3/users => generated 248 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:51.896759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814a21-2cbb-4ce2-b768-38eedff7f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:51.897353 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220328aa-6525-428c-9154-c65ef0b196fb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:51.897458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814a21-2cbb-4ce2-b768-38eedff7f9e6 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:51.897730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814a21-2cbb-4ce2-b768-38eedff7f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:51.898005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814a21-2cbb-4ce2-b768-38eedff7f9e6 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:51.898721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78814a21-2cbb-4ce2-b768-38eedff7f9e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:51.900278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78814a21-2cbb-4ce2-b768-38eedff7f9e6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:51.977050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:52.008804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 239/477] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:51 2022] POST /identity/v3/users => generated 249 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:02:52.040443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220328aa-6525-428c-9154-c65ef0b196fb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:52.081707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:52.082981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:52.083246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:52.083548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:52.084260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:52.085701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:52.128365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.sql.core [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 23:02:52.128365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 23:02:52.128365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [parameters: {'id': 'eee3de058e1649d6883ed654a75b5a36', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 23:02:52.128365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81769) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application conn.query(q) Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result.read() Sep 09 23:02:52.146931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return ref.to_dict() Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 23:02:52.147876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application next(self.gen) Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application yield resource Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application next(self.gen) Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise self.value Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application session.commit() Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 23:02:52.149240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.session.flush() Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application compat.raise_( Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 23:02:52.150307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 23:02:52.151413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application conn.query(q) Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result.read() Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 23:02:52.152221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [parameters: {'id': 'eee3de058e1649d6883ed654a75b5a36', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:52.153626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 09 23:02:52.157921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:52.161823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 239/478] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Sep 9 23:02:51 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:52.161968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814a21-2cbb-4ce2-b768-38eedff7f9e6 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:52.255560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:52.303435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814a21-2cbb-4ce2-b768-38eedff7f9e6 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:52.353951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce326155-5e10-4f47-965b-4cc90c841416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:52.354608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326155-5e10-4f47-965b-4cc90c841416 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:52.354870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326155-5e10-4f47-965b-4cc90c841416 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:52.355134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce326155-5e10-4f47-965b-4cc90c841416 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:52.366442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce326155-5e10-4f47-965b-4cc90c841416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-ce326155-5e10-4f47-965b-4cc90c841416 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:52.371685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:52.372562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:52.373375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 240/479] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:02:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:52.379933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:52.488026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502e3ef-4b27-439e-a6c6-8f9ca0778aee None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:52.506970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:52.526101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ee68c-c2a7-41f3-93e8-f6cf1f02c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:52.526855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ee68c-c2a7-41f3-93e8-f6cf1f02c78d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:52.527126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ee68c-c2a7-41f3-93e8-f6cf1f02c78d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:52.527388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ee68c-c2a7-41f3-93e8-f6cf1f02c78d None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:52.528107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ee68c-c2a7-41f3-93e8-f6cf1f02c78d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:52.528404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ee68c-c2a7-41f3-93e8-f6cf1f02c78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:52.529764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ee68c-c2a7-41f3-93e8-f6cf1f02c78d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:52.542668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 240/480] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:53.933142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16019033-faee-4e80-abff-b7c29aa77cb8 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:53.933793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16019033-faee-4e80-abff-b7c29aa77cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:53.933793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16019033-faee-4e80-abff-b7c29aa77cb8 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:53.935346 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 241/481] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:53.940481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3ad3-b324-4728-8c27-eeace391b816 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:53.940938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3ad3-b324-4728-8c27-eeace391b816 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:53.941087 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade3ad3-b324-4728-8c27-eeace391b816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:53.967458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0ade3ad3-b324-4728-8c27-eeace391b816 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:53.968649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0ade3ad3-b324-4728-8c27-eeace391b816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:54.060173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 241/482] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:54.268694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd566f9-2782-4957-85d5-09703571caf1 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.269031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd566f9-2782-4957-85d5-09703571caf1 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.269242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd566f9-2782-4957-85d5-09703571caf1 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.271051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 242/483] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:54.282833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772a1ea-0926-4548-ac32-593b6ad5c041 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.282833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772a1ea-0926-4548-ac32-593b6ad5c041 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.282833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772a1ea-0926-4548-ac32-593b6ad5c041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.307683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9772a1ea-0926-4548-ac32-593b6ad5c041 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:54.309205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9772a1ea-0926-4548-ac32-593b6ad5c041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:54.405248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 242/484] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:54.534332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd566f9-2782-4957-85d5-09703571caf1 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:54.667944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd566f9-2782-4957-85d5-09703571caf1 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:54.705733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:54.713005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ba0ef-e6e9-41cc-83c7-ff5b898a239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:54.713647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba0ef-e6e9-41cc-83c7-ff5b898a239e None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.713919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba0ef-e6e9-41cc-83c7-ff5b898a239e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.714186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ba0ef-e6e9-41cc-83c7-ff5b898a239e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.735745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ba0ef-e6e9-41cc-83c7-ff5b898a239e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:54.737895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772a1ea-0926-4548-ac32-593b6ad5c041 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-e66ba0ef-e6e9-41cc-83c7-ff5b898a239e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:54.741599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:54.742759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:54.743963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 243/485] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:02:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:54.748746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ba0ef-e6e9-41cc-83c7-ff5b898a239e None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:54.818640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:54.885601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ba0ef-e6e9-41cc-83c7-ff5b898a239e None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:54.925878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b71bf-2b09-4118-853c-49d0ad103d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:54.926395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772a1ea-0926-4548-ac32-593b6ad5c041 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:54.926533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b71bf-2b09-4118-853c-49d0ad103d2c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.926857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b71bf-2b09-4118-853c-49d0ad103d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.927112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b71bf-2b09-4118-853c-49d0ad103d2c None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.927766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b71bf-2b09-4118-853c-49d0ad103d2c None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:54.928062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b71bf-2b09-4118-853c-49d0ad103d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:54.929448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b71bf-2b09-4118-853c-49d0ad103d2c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:54.950687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 244/486] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:02:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:02:54.951909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9116f-9271-4e79-b4e9-2387a33641ba None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.952247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9116f-9271-4e79-b4e9-2387a33641ba None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.952523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9116f-9271-4e79-b4e9-2387a33641ba None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.953977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 245/487] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:54.956009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488e36-b2a6-4b2a-b7c7-b344bbde171a None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.956286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488e36-b2a6-4b2a-b7c7-b344bbde171a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.956550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488e36-b2a6-4b2a-b7c7-b344bbde171a None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.958385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 246/488] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:54.959001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb488e36-b2a6-4b2a-b7c7-b344bbde171a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:54.975296 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece19d98-5831-4726-81cd-165241b06064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:54.975931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece19d98-5831-4726-81cd-165241b06064 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:54.976190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece19d98-5831-4726-81cd-165241b06064 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:54.976454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece19d98-5831-4726-81cd-165241b06064 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:54.986918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece19d98-5831-4726-81cd-165241b06064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-ece19d98-5831-4726-81cd-165241b06064 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:54.991898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:54.998574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 243/489] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:02:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:54.999371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d993-34e4-40bd-95ea-aa5f1395f76e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.000108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d993-34e4-40bd-95ea-aa5f1395f76e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.000108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9d993-34e4-40bd-95ea-aa5f1395f76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.019307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e2c9d993-34e4-40bd-95ea-aa5f1395f76e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:55.021577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e2c9d993-34e4-40bd-95ea-aa5f1395f76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:55.023785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.082648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb488e36-b2a6-4b2a-b7c7-b344bbde171a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.091998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 244/490] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:55.093407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc86f6b-513f-421a-a2b1-43db45dae81b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.093686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc86f6b-513f-421a-a2b1-43db45dae81b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.093952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc86f6b-513f-421a-a2b1-43db45dae81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.111895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7cc86f6b-513f-421a-a2b1-43db45dae81b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:55.113610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7cc86f6b-513f-421a-a2b1-43db45dae81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:55.133810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.135329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.136105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.136599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.137454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.152540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.153965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:55.174499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:55.175459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:55.176492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:55.176492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:55.176492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:55.180180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 247/491] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:55.181128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.203324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 245/492] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:55.204563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc86f6b-513f-421a-a2b1-43db45dae81b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.302733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd561529-2cf5-4aed-bbaa-a8ea1a33a7f6 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.306495 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.345997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.347091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.347832 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.348384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.349756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:55.350435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.351991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.354733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc86f6b-513f-421a-a2b1-43db45dae81b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.362288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 248/493] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:55.370410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.423830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.424452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.430375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.430375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.430375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:55.430375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.430375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.443788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 246/494] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:55.444710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.466080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.509111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085ab46-24b0-4082-bb68-f40ed0697a89 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.549107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.550374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.551398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.552199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.563194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b75244-39e0-4b95-9632-b174f76a5d3c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.564563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.566042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:55.582562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:55.583610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:55.584440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:55.584440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:55.584440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:55.588522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 249/495] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:55.589363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.606358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53929ffd-1f31-40d3-a515-8c79c2a6041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.606914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53929ffd-1f31-40d3-a515-8c79c2a6041b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.607188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53929ffd-1f31-40d3-a515-8c79c2a6041b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.607456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53929ffd-1f31-40d3-a515-8c79c2a6041b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.610932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.619282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53929ffd-1f31-40d3-a515-8c79c2a6041b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-53929ffd-1f31-40d3-a515-8c79c2a6041b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:55.624177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:55.625028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:55.625822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 247/496] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:55.627325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53929ffd-1f31-40d3-a515-8c79c2a6041b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.705788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89a78e-b4ab-4815-96c1-0c762dcf0fed None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.707060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.727820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53929ffd-1f31-40d3-a515-8c79c2a6041b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.739506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.740240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.740403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.740667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.748089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.749498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:02:55.762546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:55.763530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:02:55.764410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:55.764909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424f1ac-1e79-476d-89b5-ebbdeec44999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.764909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f1ac-1e79-476d-89b5-ebbdeec44999 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.765055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 250/497] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:55.765121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f1ac-1e79-476d-89b5-ebbdeec44999 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.765121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424f1ac-1e79-476d-89b5-ebbdeec44999 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.765265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.778652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424f1ac-1e79-476d-89b5-ebbdeec44999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c424f1ac-1e79-476d-89b5-ebbdeec44999 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:02:55.783549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:55.784611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 248/498] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:55.785700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424f1ac-1e79-476d-89b5-ebbdeec44999 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.844855 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.866840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a520ca-0091-44cc-b0ab-5e27d0a64226 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.884588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424f1ac-1e79-476d-89b5-ebbdeec44999 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:55.900640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a9e48-4977-4581-950f-c3f3413dff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.901236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a9e48-4977-4581-950f-c3f3413dff1f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.901512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a9e48-4977-4581-950f-c3f3413dff1f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.901782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a9e48-4977-4581-950f-c3f3413dff1f None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.902511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a9e48-4977-4581-950f-c3f3413dff1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:55.902810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a9e48-4977-4581-950f-c3f3413dff1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.904207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a9e48-4977-4581-950f-c3f3413dff1f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.916355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 251/499] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:55.917586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ff225-3fa5-4e82-b2db-7a89cc439bac None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.917720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:55.917864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ff225-3fa5-4e82-b2db-7a89cc439bac None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.918130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ff225-3fa5-4e82-b2db-7a89cc439bac None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.918487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:55.918770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:55.919043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:55.922391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 252/500] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:02:55.922518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:55.922518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:55.922675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ff225-3fa5-4e82-b2db-7a89cc439bac None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:55.922774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:55.928153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:55.931373 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 249/501] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:55.932151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.029139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ff225-3fa5-4e82-b2db-7a89cc439bac None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.031096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e820b78-7206-4d2d-b857-9cf42295006e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.064519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19ce78-d5b6-44d7-83e9-82e80aedb018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.065106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ce78-d5b6-44d7-83e9-82e80aedb018 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.065379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ce78-d5b6-44d7-83e9-82e80aedb018 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.065640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19ce78-d5b6-44d7-83e9-82e80aedb018 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.065921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.066435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19ce78-d5b6-44d7-83e9-82e80aedb018 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:56.066633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.066808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19ce78-d5b6-44d7-83e9-82e80aedb018 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.067010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.067287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.067939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:56.068240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.068334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19ce78-d5b6-44d7-83e9-82e80aedb018 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.069587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.077476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 250/502] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:56.078080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.085808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:56.105037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 253/503] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:02:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:56.106230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6b52-6498-4e2f-b08c-1c7f0e64d1cb None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.106542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6b52-6498-4e2f-b08c-1c7f0e64d1cb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.111577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6b52-6498-4e2f-b08c-1c7f0e64d1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.125113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-255c6b52-6498-4e2f-b08c-1c7f0e64d1cb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:02:56.126256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-255c6b52-6498-4e2f-b08c-1c7f0e64d1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:02:56.143102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:56.182019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408498c-0705-4c5a-bbf0-6e8320d39d93 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.193799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 254/504] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:02:56.194652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6b52-6498-4e2f-b08c-1c7f0e64d1cb None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.216628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.217681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.217961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.218404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.226028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.227528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:02:56.240280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:56.241155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:56.242278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 251/505] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:56.246624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.303611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6b52-6498-4e2f-b08c-1c7f0e64d1cb None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.330322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:56.339700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c35433-341d-41d1-a618-04609d677cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.340318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c35433-341d-41d1-a618-04609d677cee None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.340587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c35433-341d-41d1-a618-04609d677cee None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.340854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c35433-341d-41d1-a618-04609d677cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.354010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c35433-341d-41d1-a618-04609d677cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.355890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c35433-341d-41d1-a618-04609d677cee None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.359275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f9f4d1-b38f-494c-8be6-1faa1aab4981 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-62c35433-341d-41d1-a618-04609d677cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:56.371758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:56.372857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:56.378607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:56.378607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:56.378607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:56.378607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 255/506] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:56.378607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c35433-341d-41d1-a618-04609d677cee None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.390431 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:56.403591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.404214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.404489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.404758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.417008 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.417963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:56.435767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:56.436681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:56.437534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:56.437534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:56.437534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:56.437534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 252/507] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:56.437774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.498144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c35433-341d-41d1-a618-04609d677cee None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.532805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b365d1f-962c-4676-9759-bb997d397809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.533403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b365d1f-962c-4676-9759-bb997d397809 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.533680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b365d1f-962c-4676-9759-bb997d397809 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.533949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b365d1f-962c-4676-9759-bb997d397809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.539845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32716f62-6737-4eb1-af54-e4507f8394a0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.572921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.573519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.573801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.574073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.574825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:56.575127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.576563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.580528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:56.601890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b365d1f-962c-4676-9759-bb997d397809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.603361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b365d1f-962c-4676-9759-bb997d397809 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.612532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 253/508] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:56.613582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.644059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:56.703058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 256/509] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:56.703896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b365d1f-962c-4676-9759-bb997d397809 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.714887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f518fd-43c5-45b7-abca-08b21ff99dda None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.748218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.748819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.749097 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.749461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.750096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:56.750421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.751795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.764093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 254/510] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:56.764881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.782120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:56.803837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b365d1f-962c-4676-9759-bb997d397809 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.837952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.838567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.838841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.839106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.839729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:56.840022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.841362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.852671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 257/511] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:56.853466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.857617 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:56.865500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07e783-b5f3-44c5-af42-f5bc9ba7cfab None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.900153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.900705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.900978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.901246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.911036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:56.912494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:56.928262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:56.929313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:56.930421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:56.930421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:56.930421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:56.934240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 255/512] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:56.935070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:56.953794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71caf7d0-7b49-4df2-9e37-27d49a1fac4f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:56.987667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325dec3-00b5-45f6-a01c-d7fa3b1212c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:56.988781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325dec3-00b5-45f6-a01c-d7fa3b1212c1 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:56.989066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325dec3-00b5-45f6-a01c-d7fa3b1212c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:56.989338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a325dec3-00b5-45f6-a01c-d7fa3b1212c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:56.997599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.039388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106a800-90f3-4982-9805-4b7be71c6149 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.058821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325dec3-00b5-45f6-a01c-d7fa3b1212c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.060279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a325dec3-00b5-45f6-a01c-d7fa3b1212c1 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.073365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.074088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.074259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.074564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.085279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.086871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.089445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:57.101597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:57.102667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:57.103716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:57.103716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:57.103716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:57.103716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 256/513] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:57.103716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.160406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 258/514] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:57.161665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325dec3-00b5-45f6-a01c-d7fa3b1212c1 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.200945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.209113 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780504d1-6521-490c-bbcc-3e2d3a2de354 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.241693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.242307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.242605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.242868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.250143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.252097 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.259502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a325dec3-00b5-45f6-a01c-d7fa3b1212c1 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:02:57.262946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:57.264024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 257/515] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:57.265054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.293262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.293967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.294312 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.294606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.302210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.303699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:02:57.316389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:57.317456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:57.318555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 259/516] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:57.319504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.334875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.378180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101cba25-39c8-4259-96ea-2f9824343e68 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.416879 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.417565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.417888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.418024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.418160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.418863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:57.419138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.421050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.421828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f780c6-c1c1-420a-8a14-fdc4a263166c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.435035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 258/517] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:57.435851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.461397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.462001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.462277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.462576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.463209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:57.463507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.464874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.476516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 260/518] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:57.477505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.533269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09852a90-0a6a-45aa-844b-7a196f67dd20 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.549763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.573395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.574166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.574763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.575044 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.581201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e215d-70aa-49fc-87d0-f8fe37e4c406 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.585991 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.587485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:57.603223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:57.604120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:57.604944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:57.604944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:57.604944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:57.604944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 259/519] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:57.605166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.621264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.621860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.622134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.622440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.623158 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:57.623451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.624873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.641478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 261/520] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:57.642234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.659682 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.702738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3400ba-7341-4d1b-b264-2c44bd2ba2f9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.736535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.737132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.737411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.737765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.739241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:57.739544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.740971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.742089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da27d4-f155-47ed-a9a1-c3ea0071e363 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.760720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 260/521] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:57.761538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.779745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dad62-ff7a-401a-9b95-6255b7cb75b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.780347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dad62-ff7a-401a-9b95-6255b7cb75b9 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.780621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dad62-ff7a-401a-9b95-6255b7cb75b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.780889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dad62-ff7a-401a-9b95-6255b7cb75b9 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.781519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dad62-ff7a-401a-9b95-6255b7cb75b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.782979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dad62-ff7a-401a-9b95-6255b7cb75b9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.787880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.857940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f0b17-11e1-4575-98c1-ab9e5c33e127 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:57.858498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 262/522] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:02:57 2022] POST /identity/v3/users => generated 247 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:02:57.859313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dad62-ff7a-401a-9b95-6255b7cb75b9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.906311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:57.907002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:57.907276 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:57.907545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:57.915385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:57.916907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:02:57.930774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:57.931705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:57.932605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 261/523] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:57.933093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:57.956134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:57.990791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dad62-ff7a-401a-9b95-6255b7cb75b9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.031039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.036592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.037460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.037732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.038001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.038688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:58.038985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.040356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.046551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed433d-e0d8-4e29-b0cb-a3b1f1224734 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.054977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 263/524] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:58.055335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.089230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8c3b9-5cb3-4e9f-b89f-15606e8c605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.090073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8c3b9-5cb3-4e9f-b89f-15606e8c605b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.090547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8c3b9-5cb3-4e9f-b89f-15606e8c605b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.090952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8c3b9-5cb3-4e9f-b89f-15606e8c605b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.161378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8c3b9-5cb3-4e9f-b89f-15606e8c605b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.163090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8c3b9-5cb3-4e9f-b89f-15606e8c605b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.172214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14428895-a7e5-4297-a2bc-ee59c950db84 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.208881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.214494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33369f-2089-4118-8c3b-c4bb7b89dc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.215183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33369f-2089-4118-8c3b-c4bb7b89dc42 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.215911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33369f-2089-4118-8c3b-c4bb7b89dc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.216251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33369f-2089-4118-8c3b-c4bb7b89dc42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.251050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.269083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 262/525] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:58.269083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8c3b9-5cb3-4e9f-b89f-15606e8c605b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.296508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33369f-2089-4118-8c3b-c4bb7b89dc42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.298371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33369f-2089-4118-8c3b-c4bb7b89dc42 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.372117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8c3b9-5cb3-4e9f-b89f-15606e8c605b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.406551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.407143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.407414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.407678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.408354 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:58.408648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.410651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.424389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 264/526] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:02:58.424389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33369f-2089-4118-8c3b-c4bb7b89dc42 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.432605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 263/527] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:58.434508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.457584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.486254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.541126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c8e64-fc49-4267-90f8-888be735c0b6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.544128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33369f-2089-4118-8c3b-c4bb7b89dc42 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.574883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.575515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.575792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.576061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.579112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.579727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.580004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.580577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.587557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.587846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.589006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.589254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:02:58.601865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:58.602756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:58.603576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:02:58.603576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:02:58.603576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:02:58.603576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:02:58.603576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:02:58.603576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:02:58.603576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:02:58.603576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:58.604430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:58.605284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:58.606124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:58.606124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:58.606124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:58.607904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 265/528] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:58.608590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.609653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 264/529] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:58.610505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.711012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.713834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:58.714782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b53ab-1b52-4d36-bf97-d4a4a0b3862b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.724454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e526f82-8879-494c-ae03-e84eadb869e2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.751570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.752158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.752440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.752712 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.753439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c3bec982c8474c349ca4901039844d26) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:58.753740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.755593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.762488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 265/530] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/role_assignments?user.id=c3bec982c8474c349ca4901039844d26&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:58.763344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.773829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.774862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.775326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.775776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.788656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.790549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:02:58.808335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:58.809276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:02:58.810111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:02:58.810111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:02:58.810111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:58.817176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 266/531] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:58.818468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.907162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784647cb-cc7e-43f4-a191-551ff6936884 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.946209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.946592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.946867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.947123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.947942 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:58.949069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.949801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.952325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd836e6-7019-4cc7-8591-77c631de6389 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:58.968771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 266/532] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:58.969661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:58.990436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:58.991076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:58.991402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:58.991675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:58.992321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:58.992629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:58.994072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:58.995600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:59.021582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 267/533] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:59.021582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.022687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:59.075438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e05e5-a813-41d8-bd2c-dcc0ecbbb44e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.111436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.112281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.112878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.113336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.114641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:59.115075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.116562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.128810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 267/534] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:59.130155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.136984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254e46e-fe31-4ae9-93c8-0d4c3d403ad9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.173616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.174249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.174419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.174685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.175468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c1842c116caa4a5d9d75c8605eeec0aa) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:59.175834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.177967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.190380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 268/535] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:59 2022] GET /identity/v3/role_assignments?user.id=c1842c116caa4a5d9d75c8605eeec0aa&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:59.190380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.265700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:59.306214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224dc513-e4ee-4f4e-8557-d34e6c4cd092 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.335127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:59.391147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.392214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.392595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.392969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.398468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab13728-e0c4-43b9-801f-9291a280746c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.403929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.405495 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:02:59.426892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:59.427801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:59.428619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:02:59.428619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:02:59.428619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:02:59.428619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:02:59.428619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:02:59.428619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:02:59.428619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:02:59.428619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:02:59.430989 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 268/536] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:59.439934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.465762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.466610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.466983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.467388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.485121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.486669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:02:59.512601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:02:59.513501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:02:59.514317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:02:59.514317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:02:59.514317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:02:59.514317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:02:59.514317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:02:59.514317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:02:59.514317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:02:59.514317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:02:59.520465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 269/537] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:02:59.521336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.648519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bb365-332e-4a90-b29a-1c244de01e15 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.717032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.717655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.717930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.718201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.721035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:59.721364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.722898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.729741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de57fc9-15a6-413f-a631-a6be94c33a91 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.745405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 269/538] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:59.751555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.755726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:59.824078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:02:59.833938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.834869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.835358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.835785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.836689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:59.837145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.838936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.859796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 270/539] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:02:59.867541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:02:59.919012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce256f89-5d15-4db3-8de8-09433071e019 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:02:59.962487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e70c9-0a11-4de5-92f8-6959aa4c94cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:02:59.963582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e70c9-0a11-4de5-92f8-6959aa4c94cd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:02:59.964053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e70c9-0a11-4de5-92f8-6959aa4c94cd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:02:59.964514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e70c9-0a11-4de5-92f8-6959aa4c94cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:02:59.965689 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e70c9-0a11-4de5-92f8-6959aa4c94cd None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=a9da2d7ef80f4622b58c9041b65df992) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:02:59.966254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e70c9-0a11-4de5-92f8-6959aa4c94cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:02:59.967956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e70c9-0a11-4de5-92f8-6959aa4c94cd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:02:59.976330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 270/540] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:59 2022] GET /identity/v3/role_assignments?user.id=a9da2d7ef80f4622b58c9041b65df992&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:00.089757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340fa33-1d14-49aa-bb08-f2fe40f7f299 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:00.152386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd5333-c134-40cc-843d-93d977957fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:00.153027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd5333-c134-40cc-843d-93d977957fbf None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:00.153306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd5333-c134-40cc-843d-93d977957fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:00.153578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd5333-c134-40cc-843d-93d977957fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:00.154354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd5333-c134-40cc-843d-93d977957fbf None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c54fab93990146e0bb746abfd0843444) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:00.154674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd5333-c134-40cc-843d-93d977957fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:00.156192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd5333-c134-40cc-843d-93d977957fbf None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:00.167584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 271/541] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:02:59 2022] GET /identity/v3/role_assignments?user.id=c54fab93990146e0bb746abfd0843444&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:00.394225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc1945-4f1f-494e-b66b-535d80c61431 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:00.395335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc1945-4f1f-494e-b66b-535d80c61431 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:00.395858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc1945-4f1f-494e-b66b-535d80c61431 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:00.397409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 271/542] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:00.402974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26b1ea-1381-4bf0-b557-27878b155d47 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:00.403406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26b1ea-1381-4bf0-b557-27878b155d47 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:00.403573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e26b1ea-1381-4bf0-b557-27878b155d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:00.432782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-8e26b1ea-1381-4bf0-b557-27878b155d47 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:00.434230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8e26b1ea-1381-4bf0-b557-27878b155d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:00.495521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:00.521022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 272/543] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:00.921420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc1945-4f1f-494e-b66b-535d80c61431 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.002881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:01.047078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc1945-4f1f-494e-b66b-535d80c61431 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:01.099818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e889c8-e163-4d84-864f-a5ab98f28e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:01.100465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e889c8-e163-4d84-864f-a5ab98f28e3d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.100809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e889c8-e163-4d84-864f-a5ab98f28e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.101082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e889c8-e163-4d84-864f-a5ab98f28e3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.112496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e889c8-e163-4d84-864f-a5ab98f28e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-28e889c8-e163-4d84-864f-a5ab98f28e3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:01.117949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:01.118858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:01.119646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 272/544] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:01.124918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26b1ea-1381-4bf0-b557-27878b155d47 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.275665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a792-9b07-465b-bf5f-ef206becd79e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.275974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a792-9b07-465b-bf5f-ef206becd79e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.276132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471a792-9b07-465b-bf5f-ef206becd79e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.277419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 273/545] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:01.282496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9084-64aa-4f2c-a1d6-905d4158803c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.282996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9084-64aa-4f2c-a1d6-905d4158803c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.283484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f9084-64aa-4f2c-a1d6-905d4158803c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.284710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e26b1ea-1381-4bf0-b557-27878b155d47 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:01.324167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:01.324766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.325039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.325303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.325956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:01.326249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:01.327710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:01.327977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-326f9084-64aa-4f2c-a1d6-905d4158803c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:01.329384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-326f9084-64aa-4f2c-a1d6-905d4158803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:01.336869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 273/546] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:01.342201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.436512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 274/547] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:03:01.445993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:01.514312 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b26c2-b95e-4fa3-be47-4ab2360b2097 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:01.553811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9523d-82f7-46f8-ae86-b9e5fa45046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:01.554381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9523d-82f7-46f8-ae86-b9e5fa45046d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.554665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9523d-82f7-46f8-ae86-b9e5fa45046d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.554928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9523d-82f7-46f8-ae86-b9e5fa45046d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.566013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9523d-82f7-46f8-ae86-b9e5fa45046d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:01.568103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9523d-82f7-46f8-ae86-b9e5fa45046d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-ffd9523d-82f7-46f8-ae86-b9e5fa45046d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:01.582994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:01.583902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:01.585224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:01.585224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:01.585224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:01.585224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 274/548] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:01.590018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f9084-64aa-4f2c-a1d6-905d4158803c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.683403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:01.731911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f9084-64aa-4f2c-a1d6-905d4158803c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:01.771295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44607539-92d4-4a30-a015-1c41344c44bb None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.771667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44607539-92d4-4a30-a015-1c41344c44bb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.772014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44607539-92d4-4a30-a015-1c41344c44bb None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.773690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 275/549] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:01.778965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d4d6-ceed-4e20-bb2e-92537ee52ac2 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.779321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d4d6-ceed-4e20-bb2e-92537ee52ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.779658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9d4d6-ceed-4e20-bb2e-92537ee52ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.787140 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:01.787884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:01.788158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:01.789146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:01.789146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:01.789459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:01.790851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:01.799098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-bec9d4d6-ceed-4e20-bb2e-92537ee52ac2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:01.800436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bec9d4d6-ceed-4e20-bb2e-92537ee52ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:01.814709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 275/550] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:01.821624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.890719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 276/551] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:01.892476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9d4d6-ceed-4e20-bb2e-92537ee52ac2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:01.946005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:01.985557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32b408-7153-49ee-84f6-d71ac846e104 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.030694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.059611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.060548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.061006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.061487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.072634 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9d4d6-ceed-4e20-bb2e-92537ee52ac2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.079426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.081386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:02.107964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:02.108921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:02.109898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:02.112592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 276/552] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:02.118739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.120064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2b468-f8f0-4bf3-9fbb-dd72c9d8ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.120709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2b468-f8f0-4bf3-9fbb-dd72c9d8ef68 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.120988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2b468-f8f0-4bf3-9fbb-dd72c9d8ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.121260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2b468-f8f0-4bf3-9fbb-dd72c9d8ef68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.139513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2b468-f8f0-4bf3-9fbb-dd72c9d8ef68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-94e2b468-f8f0-4bf3-9fbb-dd72c9d8ef68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:02.144638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:02.145592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:02.146445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:02.150477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 277/553] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:02.161557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2b468-f8f0-4bf3-9fbb-dd72c9d8ef68 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.257936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af58ad-5f58-4a48-8cb9-a13ba9e2cf4e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.276072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.298655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2b468-f8f0-4bf3-9fbb-dd72c9d8ef68 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.300589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.304370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.304748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.305097 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.305981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:02.306368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.308583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.333870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 277/554] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:02.333870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.348248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.348953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.349229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.349495 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.350253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:02.350590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.351961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.362238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 278/555] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:02.363961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.372189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.480159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c141421-aa96-4fe0-b702-35780f89b0c9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.490485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a809b-18b2-42a3-8501-61b91c58d2fa None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.527118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc05a0-ed4c-4917-8b11-c78df942f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.527987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc05a0-ed4c-4917-8b11-c78df942f49d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.528331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc05a0-ed4c-4917-8b11-c78df942f49d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.528612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc05a0-ed4c-4917-8b11-c78df942f49d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.531742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fa5fc-bf2e-4939-8422-32b985bfa847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.532560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fa5fc-bf2e-4939-8422-32b985bfa847 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.533030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fa5fc-bf2e-4939-8422-32b985bfa847 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.533490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fa5fc-bf2e-4939-8422-32b985bfa847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.543225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc05a0-ed4c-4917-8b11-c78df942f49d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a8fc05a0-ed4c-4917-8b11-c78df942f49d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:02.550088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:02.551742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 278/556] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:02.552636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc05a0-ed4c-4917-8b11-c78df942f49d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.563039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.636096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fa5fc-bf2e-4939-8422-32b985bfa847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.637448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fa5fc-bf2e-4939-8422-32b985bfa847 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.669701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc05a0-ed4c-4917-8b11-c78df942f49d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.710040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.711238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.711688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.712573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.724432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.730241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:02.742483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:02.743426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:02.744307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:02.744307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:02.744307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:02.748299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 279/557] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:02.748883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.780170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 279/558] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:03:02.780170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fece3-b847-4440-b13d-1eee621c3e7d None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.780170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fece3-b847-4440-b13d-1eee621c3e7d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.780170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263fece3-b847-4440-b13d-1eee621c3e7d None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.780170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 280/559] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:02.780170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fece3-b847-4440-b13d-1eee621c3e7d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.841936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.863196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353044e8-42c9-45b6-8a28-4b0c92f0a524 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.900273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.900792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.901068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.901277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263fece3-b847-4440-b13d-1eee621c3e7d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:02.901382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.901982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:02.902280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.904238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.917757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 280/560] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:02.917757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:02.928099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:02.943943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a7d3e-cf4a-400b-99b8-f807434d1c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:02.946109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7d3e-cf4a-400b-99b8-f807434d1c9b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.946109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7d3e-cf4a-400b-99b8-f807434d1c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.946109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a7d3e-cf4a-400b-99b8-f807434d1c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.946109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a7d3e-cf4a-400b-99b8-f807434d1c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:02.946109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a7d3e-cf4a-400b-99b8-f807434d1c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:02.947556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a7d3e-cf4a-400b-99b8-f807434d1c9b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:02.962507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 281/561] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:02.963939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162073b5-c1fa-428e-b61e-63d1fd664fd0 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:02.964229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162073b5-c1fa-428e-b61e-63d1fd664fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:02.964500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162073b5-c1fa-428e-b61e-63d1fd664fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:02.985405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-162073b5-c1fa-428e-b61e-63d1fd664fd0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:02.986629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-162073b5-c1fa-428e-b61e-63d1fd664fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:03.032534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5b82-5d35-45c2-ae58-27ef8795a17a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.057385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 282/562] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:03.058225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162073b5-c1fa-428e-b61e-63d1fd664fd0 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.072907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.073551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.073830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.074101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.087081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.088544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.102820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:03.103152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:03.104039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:03.104908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:03.104908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:03.104908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:03.104908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 281/563] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:03.105213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.166183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.172463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162073b5-c1fa-428e-b61e-63d1fd664fd0 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.209699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977213c3-1175-40bc-85c4-4f2d8c88b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.210323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977213c3-1175-40bc-85c4-4f2d8c88b266 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.210481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26647c9-441c-4a21-8202-bb9f10089b6a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.210670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977213c3-1175-40bc-85c4-4f2d8c88b266 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.210880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977213c3-1175-40bc-85c4-4f2d8c88b266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.220933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977213c3-1175-40bc-85c4-4f2d8c88b266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.222360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977213c3-1175-40bc-85c4-4f2d8c88b266 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-977213c3-1175-40bc-85c4-4f2d8c88b266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:03.238788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:03.239719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:03.240582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:03.240582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:03.240582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:03.240582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 283/564] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:03.241514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80519d6e-792e-42d1-a688-d9a4e46d8063 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.241686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80519d6e-792e-42d1-a688-d9a4e46d8063 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.241940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80519d6e-792e-42d1-a688-d9a4e46d8063 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.243318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 284/565] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:03.243873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80519d6e-792e-42d1-a688-d9a4e46d8063 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.252344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.252976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.253256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.253529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.262152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.263640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:03.274799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:03.275763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 282/566] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:03.276681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.349958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.360217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80519d6e-792e-42d1-a688-d9a4e46d8063 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.379400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956b22-4084-42f5-a38a-fdce2feb0f7c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.395456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bcd72-6eed-4f51-b465-1b6e707afd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.396061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bcd72-6eed-4f51-b465-1b6e707afd26 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.396337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bcd72-6eed-4f51-b465-1b6e707afd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.396606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bcd72-6eed-4f51-b465-1b6e707afd26 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.397244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bcd72-6eed-4f51-b465-1b6e707afd26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:03.397546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bcd72-6eed-4f51-b465-1b6e707afd26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.398659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.398937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42bcd72-6eed-4f51-b465-1b6e707afd26 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.410624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 285/567] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:03.411940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668be3-ca29-4101-8d15-15f0adef5462 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.412144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668be3-ca29-4101-8d15-15f0adef5462 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.412421 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97668be3-ca29-4101-8d15-15f0adef5462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.417477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.418037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.418305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.418834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.419480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:03.419775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.421130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.430370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-97668be3-ca29-4101-8d15-15f0adef5462 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:03.431576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-97668be3-ca29-4101-8d15-15f0adef5462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:03.433774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 283/568] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:03.434593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.502319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 286/569] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:03.503745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97668be3-ca29-4101-8d15-15f0adef5462 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.534873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c19-f4e4-46ca-8dc8-e383545b83da None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.572293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.573140 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.573593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.573866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.574626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:03.574930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.576352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.593604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 284/570] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:03.594531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.604414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.622962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97668be3-ca29-4101-8d15-15f0adef5462 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.638032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.667977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0fa46-dc46-43a6-b33d-a1dc9e5cb2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.668522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0fa46-dc46-43a6-b33d-a1dc9e5cb2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.668803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0fa46-dc46-43a6-b33d-a1dc9e5cb2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.669064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0fa46-dc46-43a6-b33d-a1dc9e5cb2c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.679377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0fa46-dc46-43a6-b33d-a1dc9e5cb2c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-70b0fa46-dc46-43a6-b33d-a1dc9e5cb2c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:03.684113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:03.685010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:03.685857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 287/571] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:03.686588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0fa46-dc46-43a6-b33d-a1dc9e5cb2c6 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.709234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6e9c8-c405-408a-bbe3-a9ddd883e45f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.744676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.745294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.745567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.745926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.752738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.754797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:03.768090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:03.768973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:03.769910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 285/572] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:03.770503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.795613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0fa46-dc46-43a6-b33d-a1dc9e5cb2c6 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.829337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.830067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.830694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.830963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.837788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.839239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.841663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:03.850856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:03.858768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 288/573] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:03.859705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.866562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:03.870306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a02a1a-2c65-4822-9a8c-c39bdc0d2516 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.903553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934208bf-f864-4df0-b340-6438377d039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.904182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934208bf-f864-4df0-b340-6438377d039a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.904732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934208bf-f864-4df0-b340-6438377d039a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.905058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934208bf-f864-4df0-b340-6438377d039a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.915895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934208bf-f864-4df0-b340-6438377d039a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.917324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934208bf-f864-4df0-b340-6438377d039a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-934208bf-f864-4df0-b340-6438377d039a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:03.931616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:03.932549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:03.938150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:03.938150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:03.938150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:03.938150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 286/574] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:03.938150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934208bf-f864-4df0-b340-6438377d039a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:03.952153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2038cff-e9ef-48b4-b991-635c7b37fc56 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:03.985451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:03.986044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:03.986323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:03.986625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:03.987287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:03.987586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:03.989602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:03.997294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 289/575] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:03.998064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.037522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934208bf-f864-4df0-b340-6438377d039a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.057719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.072258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3dce2-3b50-458b-983f-87c2bea20c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.072882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3dce2-3b50-458b-983f-87c2bea20c61 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.073228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3dce2-3b50-458b-983f-87c2bea20c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.073510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3dce2-3b50-458b-983f-87c2bea20c61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.084289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3dce2-3b50-458b-983f-87c2bea20c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-59e3dce2-3b50-458b-983f-87c2bea20c61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:04.089650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:04.090809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 287/576] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:04.091913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3dce2-3b50-458b-983f-87c2bea20c61 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.099593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48f38a-3fff-4190-9186-cc55589c6b2e None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.103467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.135972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.136644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.136848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.137117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.137802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.138103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.139552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.151470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 290/577] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.152255 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.191577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3dce2-3b50-458b-983f-87c2bea20c61 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.225021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.225620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.226060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.226332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.227070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.227370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.228818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.246107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 288/578] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.246940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.254506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e80d7e-c82e-4b8c-94bc-19cd94b6651b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.276974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.288687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.289331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.289600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.289866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.290528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.290824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.292160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.304513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 291/579] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.305306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.312837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.347382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd7b7b-43e5-4535-9165-441b84ba5b5a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.381119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.381735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.382011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.382277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.392428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.393911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.407568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315378b-b7f8-4d8e-b588-3e4a6e7db033 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:04.407804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:04.408875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:04.410913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:04.410913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:04.410913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:04.410913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 289/580] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:04.410913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.448078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.448594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.448866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.449137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.449783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.450077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.451450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.460328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 292/581] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.461094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.507088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.511459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71bfca8-897a-4805-8728-196bffe9909f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.530696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.544568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.545998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.546551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.546907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.547950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=135696b09c2c4f809c49e0af200f32de) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.548253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.549682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.555784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 290/582] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/role_assignments?user.id=135696b09c2c4f809c49e0af200f32de&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.556371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.569540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259024ae-1591-4270-ac74-ad22157ef017 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.608889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.609711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.610056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.610432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.624353 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.625966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:04.643625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:04.644678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:04.645722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:04.645722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:04.645722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:04.645722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 293/583] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:04.645722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.690293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a511b7-74ac-4dfa-bb6b-73a0865191f1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.729942 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.730538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.730815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.731083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.738615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.740040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:04.752438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:04.753348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:04.754554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:04.754554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:04.754554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:04.754554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:04.754554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:04.754554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:04.754554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:04.754554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:04.756457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 291/584] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:04.756871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.757756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.766843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f53df-1437-4a20-92cd-a04af15ac4d4 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.806395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.807003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.807286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.807549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.807650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:04.808190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.808496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.809847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.822287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 294/585] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.823151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.872641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc525b-91f2-4b1a-8b80-436005d0860d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.913733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.914566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.914845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.915114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.927040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.928604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.938297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d616a-8264-4406-a80a-500080b16a23 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:04.943568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:04.944506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:04.945390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:04.945390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:04.945390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:04.949365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 292/586] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:04.949994 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:04.976580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:04.977430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:04.977877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:04.978290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:04.979094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:04.979563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:04.981115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:04.998501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 295/587] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:04.999543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.019150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.052999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.059487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45935fec-c5ce-423b-8235-b9cdbf1d095b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.103538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.104141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.104415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.104684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.105372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:05.105674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.107139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.111128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00293be8-6d44-4de3-a3de-963da0789e5c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.123274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 293/588] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:05.123274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.154229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.155136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.156268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.156552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.165502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.167221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:05.183113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:05.184080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 296/589] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:05.189898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.222147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaf15f-e08b-4460-8eeb-eef484175ca7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.257859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.258415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.258696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.258963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.259637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:05.259936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.261303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.274422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 294/590] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:05.275025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.283954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.295159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.309351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ee073-1426-491e-b0fd-5ab90adf0cab None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.343464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.344092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.344381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.344653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.352045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.353472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:05.364875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:05.365958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:05.367199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:05.367199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:05.367199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:05.367199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:05.367199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:05.367199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:05.367199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:05.367199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:05.370660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 297/591] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:05.371479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.386823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40859d42-c5ef-4e06-b7fb-6f941ef20783 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.430527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29fdc4-2719-417d-a716-15025d85f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.431200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29fdc4-2719-417d-a716-15025d85f5df None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.431477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29fdc4-2719-417d-a716-15025d85f5df None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.431743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29fdc4-2719-417d-a716-15025d85f5df None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c3bec982c8474c349ca4901039844d26/roles/8dc6b765036246cfaf05b6553832c86a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.447437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29fdc4-2719-417d-a716-15025d85f5df None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=c3bec982c8474c349ca4901039844d26, role_id=8dc6b765036246cfaf05b6553832c86a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.448875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29fdc4-2719-417d-a716-15025d85f5df None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.483541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292869f-fc82-427b-b992-ccc53f8741f6 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.516519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 295/592] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:05 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c3bec982c8474c349ca4901039844d26/roles/8dc6b765036246cfaf05b6553832c86a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:05.517459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29fdc4-2719-417d-a716-15025d85f5df None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.523451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.524176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.524438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.524706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.525675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:05.525979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.527482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.530977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.559968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 298/593] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:05.559968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.623167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.667820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29fdc4-2719-417d-a716-15025d85f5df None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.708883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44478c79-1c44-4e40-aa9e-27b475b82204 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.725237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.726280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.726707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.727070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.736452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.738108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:05.751332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:05.752322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 296/594] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:05.753554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.764075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.764688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.764965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.765247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.765949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:05.766249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.767760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.781178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 299/595] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:05.782403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.852002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.872072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0578f-14a4-41a2-a52b-07c4987e5352 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.899285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:05.900415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb13893-30e7-4b35-aa0e-af28289a5cc2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:05.909436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.909931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.910893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.916490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.923773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.925413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.939692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:05.940601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:05.941060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:05.942052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:05.942805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:05.943738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:05.944936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:05.944936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:05.944936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:05.944936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 297/596] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:05.945631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:05.945958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:05.946555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:05.948247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:05.967337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 300/597] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:05.968509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.056859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f3645-52e1-4175-93e8-a4be27a3ba79 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.072051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00b70-87ec-4fd5-a777-dbd7e5c680a2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.086944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.094637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e473a-484f-435e-8d9c-c0717ac19455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.095351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e473a-484f-435e-8d9c-c0717ac19455 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.095988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e473a-484f-435e-8d9c-c0717ac19455 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.095988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e473a-484f-435e-8d9c-c0717ac19455 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c1842c116caa4a5d9d75c8605eeec0aa/roles/8dc6b765036246cfaf05b6553832c86a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.110647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8999e-608c-4325-901f-068c7592311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.111710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8999e-608c-4325-901f-068c7592311e None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.111821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e473a-484f-435e-8d9c-c0717ac19455 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=c1842c116caa4a5d9d75c8605eeec0aa, role_id=8dc6b765036246cfaf05b6553832c86a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.112287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8999e-608c-4325-901f-068c7592311e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.112738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8999e-608c-4325-901f-068c7592311e None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.114171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e473a-484f-435e-8d9c-c0717ac19455 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.114652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8999e-608c-4325-901f-068c7592311e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:06.115457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8999e-608c-4325-901f-068c7592311e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.117302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8999e-608c-4325-901f-068c7592311e None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.129507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 301/598] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:06.130990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8999e-608c-4325-901f-068c7592311e None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.188229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 298/599] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:05 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c1842c116caa4a5d9d75c8605eeec0aa/roles/8dc6b765036246cfaf05b6553832c86a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:06.189845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e473a-484f-435e-8d9c-c0717ac19455 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.257833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.312917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8999e-608c-4325-901f-068c7592311e None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.349325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e473a-484f-435e-8d9c-c0717ac19455 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.390957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.391965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.392442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.392907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.401538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.404017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:06.418255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:06.419144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:06.420010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:06.420010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:06.420010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:06.420010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:06.420010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:06.420010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:06.420010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:06.420010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:06.422489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 299/600] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:06.425927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.432534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.432704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.433558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.434025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.450162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.454716 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:06.474515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:06.475544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:06.476435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:06.476435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:06.476435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:06.479044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 302/601] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:06.484497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.524551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.545855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.558677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40d88-48de-4e06-bcd2-65f177a918ac None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.617073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0f718-7c0c-4a93-8d1c-273426b6e661 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.626549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.627365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.627819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.628094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.629420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:06.629817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.631378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.656041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.656875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.657292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.657701 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.658221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 300/602] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:06.658672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:06.659113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.661151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.664592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.674513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 303/603] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:06.680867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.805314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712280d6-db20-4671-97ba-0af4c600c567 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.845787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9f590-3d5a-4b3b-b77c-bc6a53790239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.847122 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9f590-3d5a-4b3b-b77c-bc6a53790239 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.847715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9f590-3d5a-4b3b-b77c-bc6a53790239 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.848383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9f590-3d5a-4b3b-b77c-bc6a53790239 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/a9da2d7ef80f4622b58c9041b65df992/roles/8dc6b765036246cfaf05b6553832c86a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.853224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.868866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9f590-3d5a-4b3b-b77c-bc6a53790239 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=a9da2d7ef80f4622b58c9041b65df992, role_id=8dc6b765036246cfaf05b6553832c86a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.870624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9f590-3d5a-4b3b-b77c-bc6a53790239 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.882398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acce51-1ed2-4b64-a54e-ff7a05571c4a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:06.920115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c040305-629f-476f-868c-1c37b6a5a05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:06.920817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c040305-629f-476f-868c-1c37b6a5a05a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.921195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c040305-629f-476f-868c-1c37b6a5a05a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.921382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c040305-629f-476f-868c-1c37b6a5a05a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.930007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c040305-629f-476f-868c-1c37b6a5a05a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:06.931517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c040305-629f-476f-868c-1c37b6a5a05a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-1c040305-629f-476f-868c-1c37b6a5a05a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:06.943803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:06.944710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:06.945581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 304/604] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:06.946647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b49c1a-6da8-41e5-b151-a2f2ec7f9c2a None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.946812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b49c1a-6da8-41e5-b151-a2f2ec7f9c2a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.949321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b49c1a-6da8-41e5-b151-a2f2ec7f9c2a None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.949321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 305/605] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:06.950124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b49c1a-6da8-41e5-b151-a2f2ec7f9c2a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:06.967102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 301/606] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:06 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/a9da2d7ef80f4622b58c9041b65df992/roles/8dc6b765036246cfaf05b6553832c86a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:03:06.968595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d2c2-baad-4d9b-aacf-8557ce4e65ba None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:06.969200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d2c2-baad-4d9b-aacf-8557ce4e65ba None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:06.969671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d2c2-baad-4d9b-aacf-8557ce4e65ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:06.977819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:06.992745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1812d2c2-baad-4d9b-aacf-8557ce4e65ba None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:06.994265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1812d2c2-baad-4d9b-aacf-8557ce4e65ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:07.138632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b49c1a-6da8-41e5-b151-a2f2ec7f9c2a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:07.168619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 302/607] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:07.186655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:07.187269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.187544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.187808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.188492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:07.188791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:07.190207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:07.203152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 306/608] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:07.209395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812d2c2-baad-4d9b-aacf-8557ce4e65ba None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:07.359863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812d2c2-baad-4d9b-aacf-8557ce4e65ba None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:07.399256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:07.406775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ad6d4-0aa0-4e53-99f3-0c69ea0833c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:07.407436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ad6d4-0aa0-4e53-99f3-0c69ea0833c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.407707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ad6d4-0aa0-4e53-99f3-0c69ea0833c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.407976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ad6d4-0aa0-4e53-99f3-0c69ea0833c4 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c54fab93990146e0bb746abfd0843444/roles/8dc6b765036246cfaf05b6553832c86a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.432011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ad6d4-0aa0-4e53-99f3-0c69ea0833c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=c54fab93990146e0bb746abfd0843444, role_id=8dc6b765036246cfaf05b6553832c86a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:07.433618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ad6d4-0aa0-4e53-99f3-0c69ea0833c4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:07.485613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:07.489702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:07.511061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 303/609] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:07 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c54fab93990146e0bb746abfd0843444/roles/8dc6b765036246cfaf05b6553832c86a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:07.628379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930c1bf3-98f3-48c6-a096-781125197358 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:07.681804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f22708-46e9-4780-9dae-ce2aa6c74410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:07.681998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22708-46e9-4780-9dae-ce2aa6c74410 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.682225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22708-46e9-4780-9dae-ce2aa6c74410 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.682366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f22708-46e9-4780-9dae-ce2aa6c74410 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.707838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f22708-46e9-4780-9dae-ce2aa6c74410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-58f22708-46e9-4780-9dae-ce2aa6c74410 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:07.713089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:07.714033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:07.715305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 307/610] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:07.720338 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ad6d4-0aa0-4e53-99f3-0c69ea0833c4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:07.859352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ad6d4-0aa0-4e53-99f3-0c69ea0833c4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:07.902767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0234c-aeb0-48ce-a9f2-24b133dcc191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:07.903382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0234c-aeb0-48ce-a9f2-24b133dcc191 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.903654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0234c-aeb0-48ce-a9f2-24b133dcc191 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.903917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0234c-aeb0-48ce-a9f2-24b133dcc191 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.904583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0234c-aeb0-48ce-a9f2-24b133dcc191 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:07.904877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0234c-aeb0-48ce-a9f2-24b133dcc191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:07.906249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0234c-aeb0-48ce-a9f2-24b133dcc191 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:07.919579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 304/611] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:07.925231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f22708-46e9-4780-9dae-ce2aa6c74410 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:07.961511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad19b-71e7-4df4-aeec-4f95c5099b7b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.961511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad19b-71e7-4df4-aeec-4f95c5099b7b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.961511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ad19b-71e7-4df4-aeec-4f95c5099b7b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.961990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 305/612] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:07.967811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfa409-5df8-4933-9628-0c1071674b33 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:07.968197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfa409-5df8-4933-9628-0c1071674b33 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:07.968555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bfa409-5df8-4933-9628-0c1071674b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:07.987221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-94bfa409-5df8-4933-9628-0c1071674b33 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:07.989177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-94bfa409-5df8-4933-9628-0c1071674b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:08.011748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:08.057742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f22708-46e9-4780-9dae-ce2aa6c74410 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:08.069967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:08.101403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:08.102703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.103223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.103864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.118982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:08.121482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:08.140449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:08.141424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:08.142291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:08.142291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:08.142291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:08.149036 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 308/613] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:08.150522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.150746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 306/614] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:03:08.277937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a5404-2c77-4dfa-97c5-b7ec2fe1950b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:08.301966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:08.328037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:08.328633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.328905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.329169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.329803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:08.330096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:08.331742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:08.349311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 309/615] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:08.355170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfa409-5df8-4933-9628-0c1071674b33 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.460756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.515054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bfa409-5df8-4933-9628-0c1071674b33 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:08.551725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:08.552273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.552552 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.552815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.560278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:08.561827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:08.574983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:08.576183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:08.577439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 307/616] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:08.584867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.634257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:08.684469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357afb5-740f-4cc1-8786-cc2a08b6c088 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:08.725830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b701e8-e1d7-4645-8ea8-55352623f89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:08.726719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b701e8-e1d7-4645-8ea8-55352623f89d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.726926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b701e8-e1d7-4645-8ea8-55352623f89d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.727244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b701e8-e1d7-4645-8ea8-55352623f89d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.740156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b701e8-e1d7-4645-8ea8-55352623f89d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-98b701e8-e1d7-4645-8ea8-55352623f89d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:08.746568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:08.747533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:08.748928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 310/617] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:08.751361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ad719-27c8-4872-903b-20aff58f508f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:08.753162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b701e8-e1d7-4645-8ea8-55352623f89d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.811488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:08.813025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:08.813730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:08.814763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:08.815860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:08.818944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:08.820597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:08.823537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:08.861217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 308/618] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Sep 9 23:03:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:08.862947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:08.948526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b701e8-e1d7-4645-8ea8-55352623f89d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.007238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.024675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.025939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.027031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.027669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.030142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:09.030688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.032585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de009cd8-50df-424a-be27-98bc0bd8ba6f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.033629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.049646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 311/619] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:09.056507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.092475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9a7e-96e4-4edb-af62-0ff972df03f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.093978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9a7e-96e4-4edb-af62-0ff972df03f4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.094480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9a7e-96e4-4edb-af62-0ff972df03f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.095142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9a7e-96e4-4edb-af62-0ff972df03f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.112631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc9a7e-96e4-4edb-af62-0ff972df03f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.114745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc9a7e-96e4-4edb-af62-0ff972df03f4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-05dc9a7e-96e4-4edb-af62-0ff972df03f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:09.130530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:09.131527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:09.132597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:09.132597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:09.132597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:09.132597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 309/620] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:09.133163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25028a6a-e397-4f77-8b88-baa0bd87bdfa None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.134075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25028a6a-e397-4f77-8b88-baa0bd87bdfa None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.134075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25028a6a-e397-4f77-8b88-baa0bd87bdfa None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.135242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 310/621] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:09.139030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25028a6a-e397-4f77-8b88-baa0bd87bdfa None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.178541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.183256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78e557-73f3-44d7-a697-bb58994a2b09 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.221863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0da877-8e51-420a-88f8-0f749a1fc2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.223139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0da877-8e51-420a-88f8-0f749a1fc2cf None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.224790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0da877-8e51-420a-88f8-0f749a1fc2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.225286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0da877-8e51-420a-88f8-0f749a1fc2cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.236928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0da877-8e51-420a-88f8-0f749a1fc2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.240194 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0da877-8e51-420a-88f8-0f749a1fc2cf None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.248911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25028a6a-e397-4f77-8b88-baa0bd87bdfa None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-db0da877-8e51-420a-88f8-0f749a1fc2cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:09.257903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:09.258907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:09.259759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:09.259759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:09.259759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:09.266003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 312/622] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:09.267548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1103b-e775-4da7-baa0-efd45209df5e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.267985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1103b-e775-4da7-baa0-efd45209df5e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.268450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1103b-e775-4da7-baa0-efd45209df5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.288033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.288878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.289256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.289676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.290455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:09.290890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.291319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-1ed1103b-e775-4da7-baa0-efd45209df5e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:09.292510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.292815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1ed1103b-e775-4da7-baa0-efd45209df5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:09.302245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.306224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 311/623] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:09.307213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.368814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 313/624] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:09.375455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1103b-e775-4da7-baa0-efd45209df5e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.422379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b5bf-8322-411d-8c82-b57c090ec727 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.452136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.452895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.453239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.453583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.454288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:09.454760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.456668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.469715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 312/625] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:09.479142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.511161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.545871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1103b-e775-4da7-baa0-efd45209df5e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.600982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.608231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.613505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.613505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.613505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.617320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.619066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:09.633533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:09.634527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:09.635563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 314/626] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:09.642729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.693517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566726b4-23f3-4aa4-81d4-2b2a8431955a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.759201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4ae48-22c2-491e-868f-4bb924a097cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.760755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4ae48-22c2-491e-868f-4bb924a097cb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.761102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4ae48-22c2-491e-868f-4bb924a097cb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.761467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4ae48-22c2-491e-868f-4bb924a097cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.771104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4ae48-22c2-491e-868f-4bb924a097cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.773676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4ae48-22c2-491e-868f-4bb924a097cb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f2e4ae48-22c2-491e-868f-4bb924a097cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:09.784629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:09.786176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 313/627] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:09.787420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a9b58-386e-44df-b488-7876bbd5776e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.788178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a9b58-386e-44df-b488-7876bbd5776e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.788178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a9b58-386e-44df-b488-7876bbd5776e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.793020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 314/628] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:09.793020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a9b58-386e-44df-b488-7876bbd5776e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.812588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da3ce3-a13f-4c90-9603-13c863a5e09b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.850811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.851409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.851678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.851940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.852620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:09.852935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:09.854321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:09.864304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.882448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 315/629] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:09.882448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:09.901589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a9b58-386e-44df-b488-7876bbd5776e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.923279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:09.938550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62618bd0-2d40-443b-8567-c004970e9343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:09.939181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618bd0-2d40-443b-8567-c004970e9343 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.939459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618bd0-2d40-443b-8567-c004970e9343 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.939722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62618bd0-2d40-443b-8567-c004970e9343 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.953509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62618bd0-2d40-443b-8567-c004970e9343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-62618bd0-2d40-443b-8567-c004970e9343 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:09.959138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:09.960078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:09.960920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:09.965222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 315/630] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:09.966531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf671-3c1a-4ac2-9af5-620925d9d714 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:09.966817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf671-3c1a-4ac2-9af5-620925d9d714 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:09.967075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf671-3c1a-4ac2-9af5-620925d9d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:09.985149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c604b1a-7d81-40b1-ab8f-3fa4862e4387 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:09.985334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d28cf671-3c1a-4ac2-9af5-620925d9d714 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:09.986539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d28cf671-3c1a-4ac2-9af5-620925d9d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:10.025032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.025645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.025927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.026203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.026958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:10.027258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.028729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.048277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 316/631] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:03:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:10.048891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.070412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 316/632] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:10.070412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cf671-3c1a-4ac2-9af5-620925d9d714 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.120471 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.166759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ab2fb-d136-43df-afea-a604af7194e8 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.193286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cf671-3c1a-4ac2-9af5-620925d9d714 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.202296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7034301-7c61-4b83-a481-8afb79fd6fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.202975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7034301-7c61-4b83-a481-8afb79fd6fe2 None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.203249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7034301-7c61-4b83-a481-8afb79fd6fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.203518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7034301-7c61-4b83-a481-8afb79fd6fe2 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/135696b09c2c4f809c49e0af200f32de/roles/8dc6b765036246cfaf05b6553832c86a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.206968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.218862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7034301-7c61-4b83-a481-8afb79fd6fe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=135696b09c2c4f809c49e0af200f32de, role_id=8dc6b765036246cfaf05b6553832c86a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.220351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7034301-7c61-4b83-a481-8afb79fd6fe2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.232174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.232776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.233074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.233341 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.234004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:10.234294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.235656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.244111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 317/633] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:10.244706 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.301418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 317/634] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:10 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/135696b09c2c4f809c49e0af200f32de/roles/8dc6b765036246cfaf05b6553832c86a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:10.303073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599cbf8-ae47-4af4-ad5f-2f90b62066ac None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.303358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599cbf8-ae47-4af4-ad5f-2f90b62066ac None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.303685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599cbf8-ae47-4af4-ad5f-2f90b62066ac None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.309581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 318/635] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:10.309581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599cbf8-ae47-4af4-ad5f-2f90b62066ac None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.380172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb5df2-f087-4ebb-a320-c1986b7feb22 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.416346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75a657-d032-41cc-b720-c3b519112e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.416918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75a657-d032-41cc-b720-c3b519112e9f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.417264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75a657-d032-41cc-b720-c3b519112e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.417436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75a657-d032-41cc-b720-c3b519112e9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.450303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599cbf8-ae47-4af4-ad5f-2f90b62066ac None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.462705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.463309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.494154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75a657-d032-41cc-b720-c3b519112e9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.494372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112885a-e453-4d5e-b665-b20bc5ecda8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.494929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112885a-e453-4d5e-b665-b20bc5ecda8a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.495205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112885a-e453-4d5e-b665-b20bc5ecda8a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.495477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112885a-e453-4d5e-b665-b20bc5ecda8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.495577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75a657-d032-41cc-b720-c3b519112e9f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.507342 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112885a-e453-4d5e-b665-b20bc5ecda8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.508945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112885a-e453-4d5e-b665-b20bc5ecda8a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-8112885a-e453-4d5e-b665-b20bc5ecda8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:10.526244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:10.530955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:10.531844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:10.531844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:10.531844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:10.531844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 319/636] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:10.531844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726a31-b758-436b-b7dc-30bc886cf211 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.531844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726a31-b758-436b-b7dc-30bc886cf211 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.531844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726a31-b758-436b-b7dc-30bc886cf211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.551330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-b2726a31-b758-436b-b7dc-30bc886cf211 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:10.552555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b2726a31-b758-436b-b7dc-30bc886cf211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:10.617082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 318/637] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:03:10.617082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75a657-d032-41cc-b720-c3b519112e9f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.627643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 320/638] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:10.631692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726a31-b758-436b-b7dc-30bc886cf211 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.711992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.739184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75a657-d032-41cc-b720-c3b519112e9f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.753434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.767355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726a31-b758-436b-b7dc-30bc886cf211 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.777122 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6381a53-5188-4db1-bfde-8daa670ffc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.777953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6381a53-5188-4db1-bfde-8daa670ffc64 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.778334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6381a53-5188-4db1-bfde-8daa670ffc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.778742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6381a53-5188-4db1-bfde-8daa670ffc64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.790369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6381a53-5188-4db1-bfde-8daa670ffc64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a6381a53-5188-4db1-bfde-8daa670ffc64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:10.796173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:10.797321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:10.798385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:10.800136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 319/639] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:10.806043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6381a53-5188-4db1-bfde-8daa670ffc64 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.819836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.821428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.822108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.822559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.823358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:10.824057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.826354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.843743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 321/640] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:10.844789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:10.919959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6381a53-5188-4db1-bfde-8daa670ffc64 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.949922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:10.956181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:10.963316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:10.963316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:10.963316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:10.966709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1ffef-f3b0-4f80-9150-f92ab17b4d7f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:10.969443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:10.970930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:10.989282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:10.990398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:10.991319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:10.991319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:10.991319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:10.991319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 320/641] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:10.992334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.010308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.010979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.011246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.011520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.012179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:11.012481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.014355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.023020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 322/642] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:11.023837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.040393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:11.108726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4a7c9-2f2e-4ef2-91ee-bc7bc3b136b6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.135439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e50db-a9c8-44b3-8558-b92d3d0e4d6c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.152113 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.153032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.153311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.153583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.164242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.166266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.169090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c8f40-0d7b-4aba-9b7c-d097d60c56b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.169713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c8f40-0d7b-4aba-9b7c-d097d60c56b6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.169985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c8f40-0d7b-4aba-9b7c-d097d60c56b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.170252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c8f40-0d7b-4aba-9b7c-d097d60c56b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:11.180933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:11.182278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 09 23:03:11.183529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:11.184094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 321/643] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:11.184498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.185489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c8f40-0d7b-4aba-9b7c-d097d60c56b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-4d4c8f40-0d7b-4aba-9b7c-d097d60c56b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:11.190467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:11.191596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 323/644] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:11.193189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c8f40-0d7b-4aba-9b7c-d097d60c56b6 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.238949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:11.280807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:11.314570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18d848-1a71-4afa-9b63-b167c5f93d89 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.315162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c8f40-0d7b-4aba-9b7c-d097d60c56b6 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.350816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.351012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.351426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.351541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.351695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.351788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.351964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.352068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.352592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:11.352892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.354279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.363702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.365155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.367113 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 322/645] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:11.368086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:11.387748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:11.388770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:11.390247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:11.390247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:11.390247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:11.390247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 324/646] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:11.390247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.510473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb87c9-4d94-497d-9fd0-98c1119a770e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.513388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:11.525083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99349d0-2644-4ffe-aab4-70882841142c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.552412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.553029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.553310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.553532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:11.553660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.554307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:11.554650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.556097 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.567654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.569685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.569685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.569685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.569685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:11.569973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.571404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.580200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 323/647] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:11.580200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.585252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 325/648] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:11.589901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.691735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f40b4-72d0-4020-9a68-a6357781c2e3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.703709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34115b-42ab-42fd-b690-e96bb21e6d4a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.732544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.733514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.733791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.734061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.741757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.744746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.744862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.745216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.745502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.745770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.746441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:11.746746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.748097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:11.756093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:11.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:11.757880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 324/649] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:11.758996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.767551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 326/650] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:11.768360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:11.795743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:11.831346 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:11.878292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9e936-6c2e-4c41-962c-9739eaca71fa None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.904220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3126f-5c4b-4145-8449-2a71a5e3bc81 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:11.917068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4de15-4208-4268-883f-839f65cf20a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.917861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4de15-4208-4268-883f-839f65cf20a2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.918135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4de15-4208-4268-883f-839f65cf20a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.918452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4de15-4208-4268-883f-839f65cf20a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.957617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:11.958792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:11.959214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:11.959626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:11.973150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.975092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.991601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4de15-4208-4268-883f-839f65cf20a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:11.993056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4de15-4208-4268-883f-839f65cf20a2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:11.997205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:11.998123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:11.999051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:11.999051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:11.999051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:11.999051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 327/651] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:11.999670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.023144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.122067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.126252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9af47-edb1-4d76-bd0b-0615201d7024 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.139054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 325/652] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:03:12.139054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4de15-4208-4268-883f-839f65cf20a2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.181399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.182106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.182486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.182823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.183592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:12.183963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.185506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.204806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 328/653] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:12.206180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.261834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4de15-4208-4268-883f-839f65cf20a2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.293009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.307469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.308482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.308482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.308938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.321091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.322568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.331899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11927e-8460-4774-a294-e0a38d514ef5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:12.340921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:12.342040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 326/654] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:12.348433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.381754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.382387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.382671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.382954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.383654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:12.383961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.385592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.405977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 329/655] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:12.405977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.429211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.474506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215819e-c775-4abb-bcf0-d8dd9e216b2d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.508970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.509607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.509886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.510159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.515615 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522de65d-b57f-4f51-a089-7f16634abed3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.520978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.522461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:12.537547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:12.538876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:12.539998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:12.539998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:12.539998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:12.543454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 327/656] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:12.544142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.556420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c3846-7c19-4cca-9def-67830882273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.557140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c3846-7c19-4cca-9def-67830882273f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.557454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c3846-7c19-4cca-9def-67830882273f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.557787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c3846-7c19-4cca-9def-67830882273f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.558676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c3846-7c19-4cca-9def-67830882273f None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c3bec982c8474c349ca4901039844d26) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:12.559043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c3846-7c19-4cca-9def-67830882273f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.560519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c3846-7c19-4cca-9def-67830882273f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.574372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 330/657] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/role_assignments?user.id=c3bec982c8474c349ca4901039844d26&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:12.574372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c3846-7c19-4cca-9def-67830882273f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.590546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.686091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be2c08a-faee-49a5-a714-57cb31427277 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.699144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.720475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c3846-7c19-4cca-9def-67830882273f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.747091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.747713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.747990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.748259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.749005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:12.749510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.750876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.763038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fa6cb-361b-4e00-84dd-a21cb05d2864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.763665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fa6cb-361b-4e00-84dd-a21cb05d2864 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.763941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fa6cb-361b-4e00-84dd-a21cb05d2864 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.764214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fa6cb-361b-4e00-84dd-a21cb05d2864 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.770599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 328/658] 10.222.0.20 () {56 vars in 1139 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:12.771368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.779159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fa6cb-361b-4e00-84dd-a21cb05d2864 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.782497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fa6cb-361b-4e00-84dd-a21cb05d2864 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-752fa6cb-361b-4e00-84dd-a21cb05d2864 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:12.796587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:12.797532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:12.798509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 331/659] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:12.799115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a55e4-9118-4956-b5ef-73015620e4e1 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.799556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a55e4-9118-4956-b5ef-73015620e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.799656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a55e4-9118-4956-b5ef-73015620e4e1 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.801910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 332/660] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:12.803224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a55e4-9118-4956-b5ef-73015620e4e1 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.881611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:12.890693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35d73e-ff3f-4094-8fac-f86201cab1c1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.925344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a55e4-9118-4956-b5ef-73015620e4e1 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:12.926878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.927616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.927896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.928198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.928914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:12.929215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:12.930611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:12.944550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 329/661] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:12.945380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:12.968227 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ecf37-a166-4104-b370-b6c331adb197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:12.968828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ecf37-a166-4104-b370-b6c331adb197 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:12.969140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ecf37-a166-4104-b370-b6c331adb197 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:12.969426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ecf37-a166-4104-b370-b6c331adb197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:12.983096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:13.045418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ecf37-a166-4104-b370-b6c331adb197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.046948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ecf37-a166-4104-b370-b6c331adb197 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.054315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11797e8f-f3ba-4b70-add3-31bf77111034 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.093945 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25953d-14df-4e79-ac47-4307b2a80c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.094626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25953d-14df-4e79-ac47-4307b2a80c81 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.094905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25953d-14df-4e79-ac47-4307b2a80c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.095177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25953d-14df-4e79-ac47-4307b2a80c81 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.095931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25953d-14df-4e79-ac47-4307b2a80c81 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:13.096239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25953d-14df-4e79-ac47-4307b2a80c81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.098454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25953d-14df-4e79-ac47-4307b2a80c81 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.123165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 330/662] 10.222.0.20 () {56 vars in 1137 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:13.123165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f51672-5086-45da-b301-dd340f653d6a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.123165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f51672-5086-45da-b301-dd340f653d6a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.123165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f51672-5086-45da-b301-dd340f653d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.145271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a8f51672-5086-45da-b301-dd340f653d6a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:13.146720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a8f51672-5086-45da-b301-dd340f653d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:13.150122 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:13.222278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 331/663] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:13.223460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f51672-5086-45da-b301-dd340f653d6a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.311759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 333/664] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:03:13.312581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ecf37-a166-4104-b370-b6c331adb197 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.335966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f51672-5086-45da-b301-dd340f653d6a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.360762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:13.384298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.384938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.385523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.385797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.395931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.397398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.400734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:13.418074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:13.419125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 332/665] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:13.419996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.449050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ecf37-a166-4104-b370-b6c331adb197 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.498573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088f9fb-9c38-4e28-b191-bb096d4d721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.499546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088f9fb-9c38-4e28-b191-bb096d4d721f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.500221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088f9fb-9c38-4e28-b191-bb096d4d721f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.500637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088f9fb-9c38-4e28-b191-bb096d4d721f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.533403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c98c4-b735-48a3-abf5-9176e0b6c2c6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.572013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.572649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.573013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.573214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.573686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088f9fb-9c38-4e28-b191-bb096d4d721f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.575151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088f9fb-9c38-4e28-b191-bb096d4d721f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.585250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.586698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:13.600864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:13.602040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:13.603167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:13.603167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:13.603167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:13.606533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 333/666] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:13.607348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.624108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:13.660617 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:13.690619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 334/667] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:03:13.691377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088f9fb-9c38-4e28-b191-bb096d4d721f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.714479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8fbae-6878-4784-94cc-9d0fd60953ff None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.748887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.749533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.749808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.750071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.750782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:13.751082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.752499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.769869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 334/668] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:13.769869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.809844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088f9fb-9c38-4e28-b191-bb096d4d721f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.848856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7dfca-aa2b-43a3-bb46-0986cad7fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.849489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7dfca-aa2b-43a3-bb46-0986cad7fbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.849759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7dfca-aa2b-43a3-bb46-0986cad7fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.850023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7dfca-aa2b-43a3-bb46-0986cad7fbe7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.860299 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7dfca-aa2b-43a3-bb46-0986cad7fbe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-efa7dfca-aa2b-43a3-bb46-0986cad7fbe7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:13.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:13.866521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:13.871559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:13.872186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b25468-d3ee-4a00-ac43-1b263a7d026a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:13.872274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 335/669] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:13.872274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7dfca-aa2b-43a3-bb46-0986cad7fbe7 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.876183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:13.884444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:13.910098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:13.910884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:13.911235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:13.911458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:13.912055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:13.912351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:13.914115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:13.925659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 335/670] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:13.926632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:13.992047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7dfca-aa2b-43a3-bb46-0986cad7fbe7 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.027221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.027857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.028113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.028379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.032142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c1f85-ca4c-4f0b-8568-e48e7342e70c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.038997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.040451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:14.056318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:14.059177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:14.060093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:14.060093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:14.060093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:14.060093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 336/671] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:14.060093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.076489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.077336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.077673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.077999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.078807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c1842c116caa4a5d9d75c8605eeec0aa) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:14.079270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.083331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.089916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 336/672] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:13 2022] GET /identity/v3/role_assignments?user.id=c1842c116caa4a5d9d75c8605eeec0aa&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:14.090823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.141258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:14.147852 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:14.213553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255b9040-f94a-43ad-822a-afb0755f96d6 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.238292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48d08a-7aac-4c7b-92bf-8dad1673a4ae None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.257993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.258895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.259353 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.259758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.260639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:14.261080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.262823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.278528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 337/673] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:14.280557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.291947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.292705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.293083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.293456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.304477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.306565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:14.322061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:14.323199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:14.324281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:14.324281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:14.324281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:14.324281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:14.324281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:14.324281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:14.324281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:14.324281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:14.330321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 337/674] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:14.330321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.445920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25433e-4558-4300-a3e8-74c50bd01431 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.463625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:14.464711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:14.469953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f6cec-9796-4c57-987c-780ea95f105c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.490925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.491920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.492255 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.492591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.493301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:14.493664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.495119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.510276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 338/675] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:14.511283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.520862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.521510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.521789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.522061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.537505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.539551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:14.559697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:14.560877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:14.562083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:14.562083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:14.562083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:14.562083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 338/676] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:14.562083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.633395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ca6b3-c7d3-4d2b-bd7e-d3b1531160c1 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.689361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.690148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.690581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.691097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.692394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:14.693183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.698730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.711819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b36f38-8200-4dc0-afa5-a66e84c52bea None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.713073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 339/677] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:14.715039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.756925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.757577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.757857 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.758124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.766380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.767935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.770152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:14.780856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:14.781862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 339/678] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:14.783374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.796108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:14.845396 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d1c95-dd66-42ae-8715-d2ec06ef732d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.881407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.882265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.882571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.882842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.883534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:14.883837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.885190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.898488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20e163-b45f-44c5-bcc8-84756ec6b80e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:14.898976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 340/679] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:14.898976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:14.934906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c0793-9d02-47be-bff0-4b42e42b02e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:14.935513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0793-9d02-47be-bff0-4b42e42b02e1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.935791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0793-9d02-47be-bff0-4b42e42b02e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.936064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c0793-9d02-47be-bff0-4b42e42b02e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.936782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c0793-9d02-47be-bff0-4b42e42b02e1 None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=a9da2d7ef80f4622b58c9041b65df992) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:14.937089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c0793-9d02-47be-bff0-4b42e42b02e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:14.938791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c0793-9d02-47be-bff0-4b42e42b02e1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:14.946376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 340/680] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/role_assignments?user.id=a9da2d7ef80f4622b58c9041b65df992&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:14.947653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a13c84-fc26-480b-9f93-4a0365f19278 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:14.947968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a13c84-fc26-480b-9f93-4a0365f19278 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:14.948211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a13c84-fc26-480b-9f93-4a0365f19278 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:14.949540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 341/681] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:14.950249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a13c84-fc26-480b-9f93-4a0365f19278 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.035808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:15.045479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33b68d-134d-49f7-af41-bc2cceb5fa12 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.059328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:15.092932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30654e-fc8b-4adf-8b55-49dd406ab59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.093926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30654e-fc8b-4adf-8b55-49dd406ab59f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.094476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30654e-fc8b-4adf-8b55-49dd406ab59f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.094952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30654e-fc8b-4adf-8b55-49dd406ab59f None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.095908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30654e-fc8b-4adf-8b55-49dd406ab59f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:15.096482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30654e-fc8b-4adf-8b55-49dd406ab59f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.099533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30654e-fc8b-4adf-8b55-49dd406ab59f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.106737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a13c84-fc26-480b-9f93-4a0365f19278 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.115281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 341/682] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:15.117145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cb997-6f36-4f0d-9d00-a4924102eb6e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.117718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cb997-6f36-4f0d-9d00-a4924102eb6e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.118186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cb997-6f36-4f0d-9d00-a4924102eb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.139536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-758cb997-6f36-4f0d-9d00-a4924102eb6e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:15.141047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-758cb997-6f36-4f0d-9d00-a4924102eb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:15.151276 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.152187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.152662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.153083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.163179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.164868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:15.178186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:15.179229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:15.180414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 342/683] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:15.181763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.241647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 342/684] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:15.250990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cb997-6f36-4f0d-9d00-a4924102eb6e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.307039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9064a-653f-4872-94f8-f57503d964c8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.368310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:15.374204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.374904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.375177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.375445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.376160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c54fab93990146e0bb746abfd0843444) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:15.376458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.380204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.399278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 343/685] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/role_assignments?user.id=c54fab93990146e0bb746abfd0843444&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:15.424211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:15.442011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cb997-6f36-4f0d-9d00-a4924102eb6e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.488058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.488958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.489257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.489531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.490372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:15.490678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.493023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.522543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 343/686] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:15.532356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.568375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.680098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54789aea-d49d-43a9-821e-355a14a47ed0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.714087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c9ba2-d010-4304-9b1c-a59f13a6366d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.719558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4010b-f403-4e4f-a761-8fe9ee1a225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.720433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4010b-f403-4e4f-a761-8fe9ee1a225b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.720913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4010b-f403-4e4f-a761-8fe9ee1a225b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.721393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4010b-f403-4e4f-a761-8fe9ee1a225b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.751275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa706763-a33f-4348-ba34-b68e353a82bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.751818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa706763-a33f-4348-ba34-b68e353a82bf None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.752108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa706763-a33f-4348-ba34-b68e353a82bf None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.752562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa706763-a33f-4348-ba34-b68e353a82bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.764678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa706763-a33f-4348-ba34-b68e353a82bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-aa706763-a33f-4348-ba34-b68e353a82bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:15.768939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:15.770913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:15.771740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 344/687] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:15.776755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa706763-a33f-4348-ba34-b68e353a82bf None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.780979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:15.799585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:15.808069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4010b-f403-4e4f-a761-8fe9ee1a225b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.809796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4010b-f403-4e4f-a761-8fe9ee1a225b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.881144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa706763-a33f-4348-ba34-b68e353a82bf None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:15.916746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:15.917314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:15.917586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:15.917851 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:15.918530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:15.918825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:15.920167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:15.928417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 344/688] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:03:15.930385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 345/689] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:15.934618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4010b-f403-4e4f-a761-8fe9ee1a225b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:15.934903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.001175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfae510-70a8-46cc-9b8a-8a3b015d75a2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.041585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.048462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4010b-f403-4e4f-a761-8fe9ee1a225b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.073009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.073880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.074295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.074737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.086742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.088407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.088713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.089628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.090005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.090414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.101755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.103391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:16.104341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:16.105424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:16.106497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:16.106497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:16.106497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:16.108162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 346/690] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:16.113747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:16.122321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:16.123547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:16.124698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:16.124698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:16.124698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:16.126128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 345/691] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:16.133200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.254367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613b068-5bf8-4456-acc5-71a75bc90830 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.254576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc08b8-a548-4b1b-b669-8b36b2ac1ae5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.277829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.283845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.284811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.285842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.286391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.291306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:16.291306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.291306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.302357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 346/692] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:16.308193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.315674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.316828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.317104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.317371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.318013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:16.318309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.319691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.327778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.345134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 347/693] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:16.350920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.419906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5df15-ae11-492f-b429-bf1d171bf1d6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.459396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.460385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.460869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.461364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.469204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.471016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.481554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbce463-3236-4d29-84b5-6d40313a9720 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:16.484705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:16.485642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:16.486550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:16.486550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:16.486550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:16.486550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:16.486550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:16.486550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:16.486550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:16.486550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:16.489295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 347/694] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:16.503574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.518869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.522649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.522929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.523203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.539629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.543758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:16.565811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:16.567034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:16.567979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 348/695] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:16.580992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.646736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.662697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.695585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe971d02-4dcb-4515-bac6-eab2795af699 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.736629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abff9e9-360e-4653-8985-79a90a1b63eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.737332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abff9e9-360e-4653-8985-79a90a1b63eb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.737673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abff9e9-360e-4653-8985-79a90a1b63eb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.738021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abff9e9-360e-4653-8985-79a90a1b63eb None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.739335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abff9e9-360e-4653-8985-79a90a1b63eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:16.739700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abff9e9-360e-4653-8985-79a90a1b63eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.741323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abff9e9-360e-4653-8985-79a90a1b63eb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.743482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b6627-d8b0-4683-86b2-ca2c42c8aed2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.754308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 348/696] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:16.758633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1fdee-993b-4183-b999-31b1e74ca58f None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.758633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1fdee-993b-4183-b999-31b1e74ca58f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.758633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1fdee-993b-4183-b999-31b1e74ca58f None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.761570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 349/697] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:16.763079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65ce3-4c56-4592-a902-cf891e2e1b20 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.763447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65ce3-4c56-4592-a902-cf891e2e1b20 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.763836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df65ce3-4c56-4592-a902-cf891e2e1b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.784290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5df65ce3-4c56-4592-a902-cf891e2e1b20 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:16.785885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5df65ce3-4c56-4592-a902-cf891e2e1b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:16.792129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.793057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.793514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.794052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.795111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:16.795596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.797326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.820448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 349/698] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:16.821605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.862483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 350/699] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 09 23:03:16.863525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df65ce3-4c56-4592-a902-cf891e2e1b20 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:16.929825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:16.936873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893578f-febd-4964-a69c-25c8bc3dcd37 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:16.975864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:16.976814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:16.977259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:16.977746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:16.978785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=135696b09c2c4f809c49e0af200f32de) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:16.979301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:16.980971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:16.988093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 350/700] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/role_assignments?user.id=135696b09c2c4f809c49e0af200f32de&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:17.102866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:17.150074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df65ce3-4c56-4592-a902-cf891e2e1b20 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.175163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.201277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f093845-53f1-4437-8afd-ca7f58490f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.201880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f093845-53f1-4437-8afd-ca7f58490f94 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.202147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f093845-53f1-4437-8afd-ca7f58490f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.204283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f093845-53f1-4437-8afd-ca7f58490f94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.287518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f093845-53f1-4437-8afd-ca7f58490f94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.289029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f093845-53f1-4437-8afd-ca7f58490f94 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.340005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2358e-cdd8-49d4-8160-889b730a8eed None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.357432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:17.385279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.386071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.386365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.386954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.402052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.403536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.404550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 351/701] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 09 23:03:17.410041 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f093845-53f1-4437-8afd-ca7f58490f94 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:17.420322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:17.421361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:17.422332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:17.422332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:17.422332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:17.422332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 351/702] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:17.431406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.454523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:17.551674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f093845-53f1-4437-8afd-ca7f58490f94 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.555811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25f1e9-961e-49d8-be14-b96a99dca1f2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.593160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.593760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.594035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.594373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.595009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:17.595301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.596678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.605991 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.606973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.607259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.607682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.609177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 352/703] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:17.614578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.621833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.623392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:17.661858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:17.664148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:17.666257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:17.666257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:17.666257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:17.666257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 352/704] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:17.671569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.698946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:17.731717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:17.777889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7dada1-81f0-4204-bf39-d29f0cd018ba None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.826336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eae7f-f534-4e37-9f62-3fcdc4d82aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.827077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eae7f-f534-4e37-9f62-3fcdc4d82aa7 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.827357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eae7f-f534-4e37-9f62-3fcdc4d82aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.827621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eae7f-f534-4e37-9f62-3fcdc4d82aa7 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.828287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eae7f-f534-4e37-9f62-3fcdc4d82aa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.829804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eae7f-f534-4e37-9f62-3fcdc4d82aa7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.876054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2fedb-6680-45cd-9bc0-b4f4b9323e71 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:17.914589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:17.915241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.915515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.915783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.916424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:17.916769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:17.918323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:17.932414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 353/705] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:17.935123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 353/706] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:03:17 2022] POST /identity/v3/users => generated 254 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:17.940194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:17.946557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d76e-c1ab-489c-8a86-54d4b2e01880 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.946865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d76e-c1ab-489c-8a86-54d4b2e01880 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.947103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3d76e-c1ab-489c-8a86-54d4b2e01880 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.948759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 354/707] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:17.968318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2aabe-e371-414d-887f-f894e7ee20b1 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:17.968654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2aabe-e371-414d-887f-f894e7ee20b1 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:17.968859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2aabe-e371-414d-887f-f894e7ee20b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:17.999441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-edc2aabe-e371-414d-887f-f894e7ee20b1 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:18.000657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-edc2aabe-e371-414d-887f-f894e7ee20b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:18.072643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:18.110175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfdba8-ee99-48ba-a11d-d98c13a581f9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.151841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c5268-a2ea-4377-9c12-d31c22a431ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.152416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c5268-a2ea-4377-9c12-d31c22a431ab None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.152688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c5268-a2ea-4377-9c12-d31c22a431ab None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.152952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c5268-a2ea-4377-9c12-d31c22a431ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.162366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c5268-a2ea-4377-9c12-d31c22a431ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.163941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c5268-a2ea-4377-9c12-d31c22a431ab None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.165068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 355/708] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-525c5268-a2ea-4377-9c12-d31c22a431ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:18.176774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:18.178884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:18.181009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 354/709] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:18.184532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2aabe-e371-414d-887f-f894e7ee20b1 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.253124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:18.298091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2aabe-e371-414d-887f-f894e7ee20b1 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.334254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcb21c-0bc2-48ae-9d1b-d8fae640ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.334927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcb21c-0bc2-48ae-9d1b-d8fae640ae6a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.335204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcb21c-0bc2-48ae-9d1b-d8fae640ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.335467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcb21c-0bc2-48ae-9d1b-d8fae640ae6a None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.336152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcb21c-0bc2-48ae-9d1b-d8fae640ae6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:18.336447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcb21c-0bc2-48ae-9d1b-d8fae640ae6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.337905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdcb21c-0bc2-48ae-9d1b-d8fae640ae6a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.351878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384b65-76bd-4fec-8605-cb23f330fc68 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.352188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 356/710] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:18.352358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384b65-76bd-4fec-8605-cb23f330fc68 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.352358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384b65-76bd-4fec-8605-cb23f330fc68 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.353802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 355/711] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:18.363753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816ad4-76e6-4d49-ad5c-5b7c0feec91c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.364181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74384b65-76bd-4fec-8605-cb23f330fc68 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.364376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816ad4-76e6-4d49-ad5c-5b7c0feec91c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.364376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e816ad4-76e6-4d49-ad5c-5b7c0feec91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.383201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3e816ad4-76e6-4d49-ad5c-5b7c0feec91c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:18.384688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3e816ad4-76e6-4d49-ad5c-5b7c0feec91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:18.480913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 357/712] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:18.503410 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74384b65-76bd-4fec-8605-cb23f330fc68 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.512184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e816ad4-76e6-4d49-ad5c-5b7c0feec91c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.541441 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:18.548167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.548870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.549214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.549550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.550696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c3bec982c8474c349ca4901039844d26) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:18.551080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.553357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.561155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 356/713] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:18 2022] GET /identity/v3/role_assignments?user.id=c3bec982c8474c349ca4901039844d26&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:18.587588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:18.662749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e816ad4-76e6-4d49-ad5c-5b7c0feec91c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.707522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.708469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.708928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.709379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] PATH_INFO: `/v3/services/network` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.710140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.711697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:18.725705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:18.726958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:18.727815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:18.729931 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 358/714] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:18 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:18.736351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.779499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.905384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1c597-a651-4e61-b20d-c350380b4f68 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.949270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:18.950055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b14a288-2330-41f8-98b0-9f1f0a90da81 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:18.950295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:18.950763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:18.951185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:18.952335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] RBAC: Adding query filter params (name=network) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:18.953645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:18.955205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:18.961580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 357/715] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:18.969940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:18.972249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:19.007418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.008076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.008358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.008631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.009223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.010622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:19.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:19.038715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:19.039569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:19.046718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 359/716] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Sep 9 23:03:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:19.047611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.049728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:19.094678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e6a9a-334a-46da-b35b-a56d6984c02d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.151393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.152074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.152301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.152544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.153241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:19.153543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.154931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.174216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 358/717] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:18 2022] GET /identity/v3/services?type=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:19.233442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5ea7a-a3d1-4168-95f1-3778bc6addf7 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.317406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fd51e-2022-402f-b508-6671f7c4f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.318038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fd51e-2022-402f-b508-6671f7c4f029 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.318311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fd51e-2022-402f-b508-6671f7c4f029 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.318705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fd51e-2022-402f-b508-6671f7c4f029 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.319363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fd51e-2022-402f-b508-6671f7c4f029 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:19.319715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fd51e-2022-402f-b508-6671f7c4f029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.321129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fd51e-2022-402f-b508-6671f7c4f029 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.332720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 360/718] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Sep 9 23:03:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:19.338588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:19.460434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbce55-c2c3-4720-ae95-21172d7b7ed7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:19.487455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:19.499633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474284a6-08ed-41ea-a587-a6af786d2bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:19.500244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474284a6-08ed-41ea-a587-a6af786d2bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.505669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474284a6-08ed-41ea-a587-a6af786d2bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.505669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474284a6-08ed-41ea-a587-a6af786d2bf9 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.505669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474284a6-08ed-41ea-a587-a6af786d2bf9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:19.505669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474284a6-08ed-41ea-a587-a6af786d2bf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:19.505669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474284a6-08ed-41ea-a587-a6af786d2bf9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:19.508374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 359/719] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Sep 9 23:03:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:19.893724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69298134-4d09-465a-bbac-4a7ced3f4593 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.894097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69298134-4d09-465a-bbac-4a7ced3f4593 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.894409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69298134-4d09-465a-bbac-4a7ced3f4593 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.896041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 361/720] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:19.901688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7447d-9f2d-4b18-8983-20b7e6ea3e31 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:19.901974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7447d-9f2d-4b18-8983-20b7e6ea3e31 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:19.902249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7447d-9f2d-4b18-8983-20b7e6ea3e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:19.931452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1ab7447d-9f2d-4b18-8983-20b7e6ea3e31 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:19.932737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1ab7447d-9f2d-4b18-8983-20b7e6ea3e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:20.014718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 360/721] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:20.472739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69298134-4d09-465a-bbac-4a7ced3f4593 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.529008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:20.590129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69298134-4d09-465a-bbac-4a7ced3f4593 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.626595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72528f9-b516-4a9d-9150-cdbd3d7d9d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.627219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72528f9-b516-4a9d-9150-cdbd3d7d9d03 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.627489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72528f9-b516-4a9d-9150-cdbd3d7d9d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.627759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72528f9-b516-4a9d-9150-cdbd3d7d9d03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.638496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72528f9-b516-4a9d-9150-cdbd3d7d9d03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-a72528f9-b516-4a9d-9150-cdbd3d7d9d03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:20.643775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:20.644770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:20.645631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 362/722] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:20.651123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7447d-9f2d-4b18-8983-20b7e6ea3e31 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.778043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7447d-9f2d-4b18-8983-20b7e6ea3e31 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:20.802871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:20.821335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:20.822116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:20.822414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:20.822684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:20.823342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:20.823731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:20.825077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:20.838437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 361/723] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:20.844025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72528f9-b516-4a9d-9150-cdbd3d7d9d03 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:20.984495 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:20.992786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72528f9-b516-4a9d-9150-cdbd3d7d9d03 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.040332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ccb31f-d305-4868-8893-9f1c33427d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.040884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccb31f-d305-4868-8893-9f1c33427d95 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.041161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccb31f-d305-4868-8893-9f1c33427d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.041426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ccb31f-d305-4868-8893-9f1c33427d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.058393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccb31f-d305-4868-8893-9f1c33427d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.059849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ccb31f-d305-4868-8893-9f1c33427d95 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-d0ccb31f-d305-4868-8893-9f1c33427d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:21.085805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:21.086919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:21.088407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:21.088407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:21.088407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:21.088407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 363/724] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:21.092796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.133221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428e65-97ef-4b3e-b06e-3dc80542e9ea None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.133540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428e65-97ef-4b3e-b06e-3dc80542e9ea None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.133765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428e65-97ef-4b3e-b06e-3dc80542e9ea None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.135397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 364/725] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:21.140413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b61eb6-3aa4-4bcb-bbda-0f4460d95b20 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.140690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b61eb6-3aa4-4bcb-bbda-0f4460d95b20 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.140962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b61eb6-3aa4-4bcb-bbda-0f4460d95b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.160188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-56b61eb6-3aa4-4bcb-bbda-0f4460d95b20 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:21.161566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-56b61eb6-3aa4-4bcb-bbda-0f4460d95b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.262313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb02cc-fa15-4377-83fd-4effe20ffaad None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.279945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 365/726] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:21.282081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68253-b5a2-4fd9-978a-bf1799635a4d None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.282610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68253-b5a2-4fd9-978a-bf1799635a4d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.283088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d68253-b5a2-4fd9-978a-bf1799635a4d None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.289064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 366/727] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:21.291791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d1b6-32f0-4c57-bbff-da0ce74b03cd None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.292018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d1b6-32f0-4c57-bbff-da0ce74b03cd None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.292281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d1b6-32f0-4c57-bbff-da0ce74b03cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.307356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.308286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.308661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.309029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.309789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:21.310184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.311675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.313019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4c54d1b6-32f0-4c57-bbff-da0ce74b03cd None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:21.314525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4c54d1b6-32f0-4c57-bbff-da0ce74b03cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:21.326470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 362/728] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:21.333021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.375400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:21.449017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ad25b-8388-4558-a72c-21cf4649d0bb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.465698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 367/729] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:21.485861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.486850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.487371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.487844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.495867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.497630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:21.510503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:21.511481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:21.512433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:21.514914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 363/730] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:21.520996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54d1b6-32f0-4c57-bbff-da0ce74b03cd None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.537260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:21.674021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54d1b6-32f0-4c57-bbff-da0ce74b03cd None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.696789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.734297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.734939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.735215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.735483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.736218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:21.736518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:21.737941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:21.774531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 368/731] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:21.780824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.864947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:21.870100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81911cb3-e126-4353-9adb-745e132bfe70 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.871693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:21.907665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60afd1-8d83-43c0-a35f-54d8934c0f4b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:21.922498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187243b-fd7c-40ad-844a-238a2489d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.923574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187243b-fd7c-40ad-844a-238a2489d8ed None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.923855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187243b-fd7c-40ad-844a-238a2489d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.924187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f187243b-fd7c-40ad-844a-238a2489d8ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:21.942088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f187243b-fd7c-40ad-844a-238a2489d8ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f187243b-fd7c-40ad-844a-238a2489d8ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:21.948859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:21.949927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:21.950861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 364/732] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:21.951557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187243b-fd7c-40ad-844a-238a2489d8ed None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:21.952681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c4cda-177d-4202-9cb9-de7f1067a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:21.953683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4cda-177d-4202-9cb9-de7f1067a2ce None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:21.954160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4cda-177d-4202-9cb9-de7f1067a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:21.954660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c4cda-177d-4202-9cb9-de7f1067a2ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.029560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c4cda-177d-4202-9cb9-de7f1067a2ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.031390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c4cda-177d-4202-9cb9-de7f1067a2ce None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.078020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f187243b-fd7c-40ad-844a-238a2489d8ed None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.106285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.113907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1a7b-1b07-4e25-9f00-c1100027a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.114783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1a7b-1b07-4e25-9f00-c1100027a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.115065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1a7b-1b07-4e25-9f00-c1100027a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.115332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c1a7b-1b07-4e25-9f00-c1100027a2a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.125905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c1a7b-1b07-4e25-9f00-c1100027a2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-cf5c1a7b-1b07-4e25-9f00-c1100027a2a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:22.131678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:22.132589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:22.133425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 365/733] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:22.134091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1a7b-1b07-4e25-9f00-c1100027a2a9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.149372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 369/734] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 09 23:03:22.149372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48244f62-4906-4cb8-a7de-a3bc9b895e37 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.149372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48244f62-4906-4cb8-a7de-a3bc9b895e37 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.149882 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48244f62-4906-4cb8-a7de-a3bc9b895e37 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.151357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 370/735] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:22.155603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48244f62-4906-4cb8-a7de-a3bc9b895e37 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.164089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.245755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c1a7b-1b07-4e25-9f00-c1100027a2a9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.270611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48244f62-4906-4cb8-a7de-a3bc9b895e37 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.279655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.280258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.280529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.280802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.281451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:22.281751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.283130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.291996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 366/736] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:22.292795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.311210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec735e4-c8b5-445d-9408-0cbc53c204a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.311805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec735e4-c8b5-445d-9408-0cbc53c204a7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.312072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec735e4-c8b5-445d-9408-0cbc53c204a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.312377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec735e4-c8b5-445d-9408-0cbc53c204a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.313026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec735e4-c8b5-445d-9408-0cbc53c204a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:22.313317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec735e4-c8b5-445d-9408-0cbc53c204a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.314687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec735e4-c8b5-445d-9408-0cbc53c204a7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.322712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 371/737] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:22.323849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1fa8b-84b9-4a8a-88ea-48d022743480 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.324125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1fa8b-84b9-4a8a-88ea-48d022743480 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.324391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1fa8b-84b9-4a8a-88ea-48d022743480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.342430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-9fd1fa8b-84b9-4a8a-88ea-48d022743480 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:22.343606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9fd1fa8b-84b9-4a8a-88ea-48d022743480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:22.369955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.377458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.397257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87615b04-8e48-497c-9984-c14abea4d0b6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.420579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 372/738] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:22.420579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf26066-d0a4-43a4-b9ce-b6225462b764 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.420579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf26066-d0a4-43a4-b9ce-b6225462b764 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.420579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf26066-d0a4-43a4-b9ce-b6225462b764 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.420579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 373/739] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:22.420579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf26066-d0a4-43a4-b9ce-b6225462b764 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.442983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.443653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.443933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.444211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.455301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.457585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:22.473800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:22.474806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:22.475694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:22.475694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:22.475694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:22.475694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 367/740] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:22.475694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.546026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf26066-d0a4-43a4-b9ce-b6225462b764 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.578426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b9c2b-6e50-4062-b5e8-7ed16e1b9d89 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.585031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab27c64-993b-400a-a062-ddced4b47554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.585651 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27c64-993b-400a-a062-ddced4b47554 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.585923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27c64-993b-400a-a062-ddced4b47554 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.586190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab27c64-993b-400a-a062-ddced4b47554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.597662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab27c64-993b-400a-a062-ddced4b47554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.599366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab27c64-993b-400a-a062-ddced4b47554 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.609034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.615431 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.616072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.616367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.616625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-7ab27c64-993b-400a-a062-ddced4b47554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:22.617837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:22.623968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:22.624854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:22.624854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:22.624854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:22.624854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 374/741] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:22.624854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a4077-e24c-4e23-8444-f1c00ce0a58b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.624854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a4077-e24c-4e23-8444-f1c00ce0a58b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.624854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a4077-e24c-4e23-8444-f1c00ce0a58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.627199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.628672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.639790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-000a4077-e24c-4e23-8444-f1c00ce0a58b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:22.641196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-000a4077-e24c-4e23-8444-f1c00ce0a58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:22.643855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:22.645327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:22.646221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:22.646221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:22.646221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:22.646221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 368/742] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:22.646475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.659328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.722233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 375/743] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 09 23:03:22.723791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a4077-e24c-4e23-8444-f1c00ce0a58b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.758635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9b37f-a974-4b12-b2f4-d63827dfcff2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.797060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.797615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.797893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.798154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.798839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:22.799145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.800504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.816528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 369/744] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:22.816528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.848895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a4077-e24c-4e23-8444-f1c00ce0a58b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.863578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.886405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.887810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.888093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.888363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.889013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:22.889314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.890903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.904708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 376/745] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:22.905490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:22.910994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:22.929373 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d81d4-b634-4a26-beb8-112996444f43 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:22.977749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:22.978394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:22.978693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:22.978939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:22.979579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:22.979881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:22.981240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:22.993089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 370/746] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:22.994038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.016941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec66e1-703e-4b93-a725-5c968f0fb70c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.050121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45729cae-caba-4c53-80d3-0f01fe140e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.050675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729cae-caba-4c53-80d3-0f01fe140e3c None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.050972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729cae-caba-4c53-80d3-0f01fe140e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.051202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45729cae-caba-4c53-80d3-0f01fe140e3c None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.051837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45729cae-caba-4c53-80d3-0f01fe140e3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.053175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45729cae-caba-4c53-80d3-0f01fe140e3c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.091844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b047521-0131-4e58-b973-59ec547c5385 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.094088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.123517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.124135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.124413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.124682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.131524 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.132947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:23.144986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:23.145853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:23.146759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 371/747] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:23.147417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.180166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 377/748] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:03:22 2022] POST /identity/v3/services => generated 240 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:23.180973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45729cae-caba-4c53-80d3-0f01fe140e3c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.258209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.263708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e839359-435f-4a37-9f9f-83f5e7599f37 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.310075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.311047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.311506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.311924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.314618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45729cae-caba-4c53-80d3-0f01fe140e3c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.321917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.323707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:23.342195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:23.343392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 372/749] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:23.344328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.364522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587a4e1-c027-4819-a9fd-49ea36c09205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.365133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a4e1-c027-4819-a9fd-49ea36c09205 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.365828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a4e1-c027-4819-a9fd-49ea36c09205 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.366099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a4e1-c027-4819-a9fd-49ea36c09205 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.367063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587a4e1-c027-4819-a9fd-49ea36c09205 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.368448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587a4e1-c027-4819-a9fd-49ea36c09205 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.371906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.433948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 378/750] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:03:23 2022] POST /identity/v3/services => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:23.438497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587a4e1-c027-4819-a9fd-49ea36c09205 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.461444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7aa65-f805-4bbf-886f-3c650e211a4f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.513406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.514078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.514458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.514793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.523502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.525071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:23.538724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:23.539932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:23.541109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:23.542163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 373/751] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:23.543030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.598703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587a4e1-c027-4819-a9fd-49ea36c09205 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.635356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.636473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.647685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.648503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.648929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.649486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.650375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:23.650823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.652402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.668029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 379/752] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:23.668880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.670480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d1e84-131a-4400-af33-0e1d703e7cdd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.711142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.711620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.711893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.712160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.713037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:23.713216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.714631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.733590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 374/753] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:23.735960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.813935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69024226-62d3-4b8f-80b5-fe544ca4b914 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.840892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af80ca-0641-40db-b10c-3096f37ca2cc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:23.862475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.864030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.864473 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.865189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.866073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:23.866791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.870270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.878290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:23.879754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:23.880231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:23.880600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:23.881418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=135696b09c2c4f809c49e0af200f32de) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:23.881837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:23.883365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:23.885363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:23.894881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 375/754] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/role_assignments?user.id=135696b09c2c4f809c49e0af200f32de&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:23.894881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.910065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 380/755] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:23.911703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:23.990686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.099913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36272d42-a062-45a6-9882-976d2d5913fc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.115314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05bfd-ed9f-4d07-bad3-93b9dfc0b0b8 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.141032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694de54-194a-4e78-bec7-42a1bc282653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.141953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694de54-194a-4e78-bec7-42a1bc282653 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.142444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694de54-194a-4e78-bec7-42a1bc282653 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.142913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694de54-194a-4e78-bec7-42a1bc282653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.159530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.161013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.161465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.162149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.177473 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.179532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:24.197579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:24.198731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:24.199613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:24.199613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:24.199613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:24.202846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 381/756] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:24.210505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.237236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.241955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694de54-194a-4e78-bec7-42a1bc282653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.243646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f694de54-194a-4e78-bec7-42a1bc282653 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.333010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.339867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b0dec-296a-4424-94f5-1c97a96526cb None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.383239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.383773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.384046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.384313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.384958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:24.385253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.388225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.413273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 382/757] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:24.423467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.452542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 376/758] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:03:24.457985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694de54-194a-4e78-bec7-42a1bc282653 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.610322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba865c-bf04-4053-bfe5-6c2e0aacaf4c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.631708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.638418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694de54-194a-4e78-bec7-42a1bc282653 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.652732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.654119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.654659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.655156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.664982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.666797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:24.688057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:24.689000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:24.689906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:24.689906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:24.689906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:24.689906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:24.689906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:24.689906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:24.689906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:24.689906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:24.693825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 383/759] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:24.700002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.715860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.716934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.717418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.717888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.723372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:24.731798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.734013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:24.751406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:24.752402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:24.753312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:24.753312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:24.753312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:24.755968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 377/760] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:24.761336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:24.890837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367ffae-d1a6-44b4-a5a8-ca36f57569e9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.938868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:24.941186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:24.941958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:24.943451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:24.944362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:24.946067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:24.947735 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:24.974354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 384/761] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:24.980074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77ee5b-91cb-440d-93c5-662d45780c57 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:24.981699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.068032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.068652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.068925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.069195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.070059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:25.070171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.071491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.073625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:25.107730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbfa52-2655-4bde-b4ba-6413d680811f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.108556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 378/762] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:25.119335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.136631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:25.154414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6567ff3-0f3c-46a8-9b43-b909fbeeceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.155438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6567ff3-0f3c-46a8-9b43-b909fbeeceb9 None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.155913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6567ff3-0f3c-46a8-9b43-b909fbeeceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.156415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6567ff3-0f3c-46a8-9b43-b909fbeeceb9 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c3bec982c8474c349ca4901039844d26/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.159730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6567ff3-0f3c-46a8-9b43-b909fbeeceb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=c3bec982c8474c349ca4901039844d26, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.163127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6567ff3-0f3c-46a8-9b43-b909fbeeceb9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.231757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 385/763] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:24 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c3bec982c8474c349ca4901039844d26/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:25.277343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62688b-b2df-42fe-bdd7-fa2467bbb336 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.334482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.335545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.336053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.336530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.344854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.346678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:25.364290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:25.365257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:25.366135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:25.366135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:25.366135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:25.366135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:25.366135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:25.366135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:25.366135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:25.366135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:25.368491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 379/764] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:25.374274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6567ff3-0f3c-46a8-9b43-b909fbeeceb9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.518136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6567ff3-0f3c-46a8-9b43-b909fbeeceb9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.549269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:25.555530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826db3ee-db3a-4e6e-864f-6ecf7ab9b890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.558283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826db3ee-db3a-4e6e-864f-6ecf7ab9b890 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.558283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826db3ee-db3a-4e6e-864f-6ecf7ab9b890 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.558283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826db3ee-db3a-4e6e-864f-6ecf7ab9b890 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.558283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826db3ee-db3a-4e6e-864f-6ecf7ab9b890 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:25.558283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826db3ee-db3a-4e6e-864f-6ecf7ab9b890 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.559133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826db3ee-db3a-4e6e-864f-6ecf7ab9b890 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.575416 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 386/765] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:25.581402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:25.701708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:25.710407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737f115-c5cc-4ac1-ae45-064a547de847 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.710914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737f115-c5cc-4ac1-ae45-064a547de847 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.711007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737f115-c5cc-4ac1-ae45-064a547de847 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.712313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 387/766] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:25.721622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d84ee-e1a0-447f-a7a6-7dbf77dd1c0e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.721907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d84ee-e1a0-447f-a7a6-7dbf77dd1c0e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.722177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d84ee-e1a0-447f-a7a6-7dbf77dd1c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.732951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcbdb3-eaf4-441d-9136-0cfeb28375aa None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:25.753540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-895d84ee-e1a0-447f-a7a6-7dbf77dd1c0e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:25.755676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-895d84ee-e1a0-447f-a7a6-7dbf77dd1c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:25.829433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288cbee1-ebd5-46cb-9248-13e69f2e1a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:25.830702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cbee1-ebd5-46cb-9248-13e69f2e1a55 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:25.831245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cbee1-ebd5-46cb-9248-13e69f2e1a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:25.831518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288cbee1-ebd5-46cb-9248-13e69f2e1a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:25.834261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cbee1-ebd5-46cb-9248-13e69f2e1a55 None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=2c8bfd34442b48a78600f568b534eaa1) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:25.834615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cbee1-ebd5-46cb-9248-13e69f2e1a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:25.836649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288cbee1-ebd5-46cb-9248-13e69f2e1a55 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:25.857462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 380/767] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:25 2022] GET /identity/v3/role_assignments?user.id=2c8bfd34442b48a78600f568b534eaa1&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:25.890957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 388/768] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:26.008107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c59a5-6510-4de0-a111-b3df1456844e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.008624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c59a5-6510-4de0-a111-b3df1456844e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.008624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c59a5-6510-4de0-a111-b3df1456844e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.010170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 381/769] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:26.015622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86577384-383d-44be-a5e2-5860c6d5dd9b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.015903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86577384-383d-44be-a5e2-5860c6d5dd9b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.016179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86577384-383d-44be-a5e2-5860c6d5dd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.040041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-86577384-383d-44be-a5e2-5860c6d5dd9b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:26.041264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-86577384-383d-44be-a5e2-5860c6d5dd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.139235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 389/770] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:26.248260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c59a5-6510-4de0-a111-b3df1456844e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.389078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c59a5-6510-4de0-a111-b3df1456844e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.447249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.447881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.448157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.448427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.449007 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.450487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.451202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86577384-383d-44be-a5e2-5860c6d5dd9b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.455229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:26.465824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:26.466854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:26.468324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 382/771] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:26.475369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.530579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:26.601655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86577384-383d-44be-a5e2-5860c6d5dd9b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.608983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b232557-69e0-41b6-8f52-0407ec7e9cf8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.646233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73050728-dbd7-46ab-9c17-a6dadfe23113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.647094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73050728-dbd7-46ab-9c17-a6dadfe23113 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.647562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73050728-dbd7-46ab-9c17-a6dadfe23113 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.647975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73050728-dbd7-46ab-9c17-a6dadfe23113 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.648145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a772d-d749-4b73-8f87-00fbd2da8aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.648776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a772d-d749-4b73-8f87-00fbd2da8aab None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.649052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a772d-d749-4b73-8f87-00fbd2da8aab None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.649262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73050728-dbd7-46ab-9c17-a6dadfe23113 None admin] RBAC: Adding query filter params (name=network) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:26.649462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a772d-d749-4b73-8f87-00fbd2da8aab None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.649832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73050728-dbd7-46ab-9c17-a6dadfe23113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.650042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a772d-d749-4b73-8f87-00fbd2da8aab None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.651433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a772d-d749-4b73-8f87-00fbd2da8aab None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.651716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73050728-dbd7-46ab-9c17-a6dadfe23113 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.656982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 383/772] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:26.658526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c9ba-2bac-46ba-8839-68c8efbcb89e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.659001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c9ba-2bac-46ba-8839-68c8efbcb89e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.659441 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1c9ba-2bac-46ba-8839-68c8efbcb89e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-617a772d-d749-4b73-8f87-00fbd2da8aab None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:26.662175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:26.663198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:26.664018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:26.665050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 384/773] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:26.665939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1c9ba-2bac-46ba-8839-68c8efbcb89e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.667333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 390/774] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:26.668565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198867-9c46-447f-beed-7b8e14922bb8 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.668854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198867-9c46-447f-beed-7b8e14922bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.669128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198867-9c46-447f-beed-7b8e14922bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.689738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e9198867-9c46-447f-beed-7b8e14922bb8 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:26.691986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e9198867-9c46-447f-beed-7b8e14922bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:26.748250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:26.764695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 391/775] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:26.766185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9198867-9c46-447f-beed-7b8e14922bb8 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.825975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1c9ba-2bac-46ba-8839-68c8efbcb89e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:26.853282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:26.893938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:26.894611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:26.894883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:26.895146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:26.895767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] RBAC: Adding query filter params (type=network) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:26.896074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:26.898705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:26.907454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 385/776] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:26.912290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:26.950391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9198867-9c46-447f-beed-7b8e14922bb8 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.021826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.022475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.022824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.023098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.023786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:27.024096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.025468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.037414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 392/777] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:27.039938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.101960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f738c-54d6-4573-af9c-1daa92f5a93c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.147306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:27.165639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.165639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.165639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.165639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.166004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] RBAC: Adding query filter params (interface=public, service_id=d60386e5031b40bc86a1110dbdc4338c, region_id=RegionOne) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:27.166004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.166004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.173307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 386/778] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Sep 9 23:03:26 2022] GET /identity/v3/endpoints?service_id=d60386e5031b40bc86a1110dbdc4338c&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:27.174537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.193997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d6ca3-beda-4fa3-8836-bf2c466428cf None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.205355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:27.236407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.237832 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.238851 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.239127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.239768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:27.240072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.241450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.248849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 393/779] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:27.250025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.304544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6073d-2892-41e2-afdd-b30da639be15 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.340941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f14e7d-0338-4848-b1fa-9165ec565d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.341997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f14e7d-0338-4848-b1fa-9165ec565d67 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.342543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f14e7d-0338-4848-b1fa-9165ec565d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.343013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f14e7d-0338-4848-b1fa-9165ec565d67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.354221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f14e7d-0338-4848-b1fa-9165ec565d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-36f14e7d-0338-4848-b1fa-9165ec565d67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:27.359909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:27.360832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:27.361742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:27.363980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90320991-cea4-471c-b475-4f7a480d7707 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.369434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 387/780] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:27.370612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f14e7d-0338-4848-b1fa-9165ec565d67 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.403646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaeabe-2692-4cd4-a723-44dd9d6b0bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.404266 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaeabe-2692-4cd4-a723-44dd9d6b0bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.404486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaeabe-2692-4cd4-a723-44dd9d6b0bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.404764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaeabe-2692-4cd4-a723-44dd9d6b0bc5 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.405375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaeabe-2692-4cd4-a723-44dd9d6b0bc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.407019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaeabe-2692-4cd4-a723-44dd9d6b0bc5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.416577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 394/781] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:27.417461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaeabe-2692-4cd4-a723-44dd9d6b0bc5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.442104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:27.459720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:27.539287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaeabe-2692-4cd4-a723-44dd9d6b0bc5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.544390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f14e7d-0338-4848-b1fa-9165ec565d67 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.580730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.582089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.582801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.583728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.584666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:27.585185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.586811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.596566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 395/782] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:27.607427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.615138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.615603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.616733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.616733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.616919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] RBAC: Adding query filter params (interface=public, service_id=36a40301114540d690087d83c0770c29, region_id=RegionOne) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:27.617130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.623229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.631349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 388/783] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/endpoints?service_id=36a40301114540d690087d83c0770c29&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:27.638082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.774099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f6be8-0be4-453a-8143-c63b70d5aa55 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.780260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:27.787102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d74741-80f7-4394-aecd-9254ace72b8d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:27.790094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:27.812591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57a245-f32b-447d-ac56-c2fc69211bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.813613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57a245-f32b-447d-ac56-c2fc69211bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.814100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57a245-f32b-447d-ac56-c2fc69211bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.814774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57a245-f32b-447d-ac56-c2fc69211bf9 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.815616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57a245-f32b-447d-ac56-c2fc69211bf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.818324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57a245-f32b-447d-ac56-c2fc69211bf9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.825625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 389/784] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:27.835765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8654d35e-e571-46d8-a000-2dcdae2eae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:27.839127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654d35e-e571-46d8-a000-2dcdae2eae12 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.839127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654d35e-e571-46d8-a000-2dcdae2eae12 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.839127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654d35e-e571-46d8-a000-2dcdae2eae12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.851313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654d35e-e571-46d8-a000-2dcdae2eae12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:27.852914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8654d35e-e571-46d8-a000-2dcdae2eae12 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-8654d35e-e571-46d8-a000-2dcdae2eae12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:27.870783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:27.871792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:27.872668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:27.872668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:27.872668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:27.872668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 396/785] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:27.877801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57a245-f32b-447d-ac56-c2fc69211bf9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:27.958765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50679285-9028-43cb-8010-c11e78588b3e None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.959093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50679285-9028-43cb-8010-c11e78588b3e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.959337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50679285-9028-43cb-8010-c11e78588b3e None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:27.960935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 397/786] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:27.972842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f752-8bf2-42bc-9755-c037b83f976e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:27.973188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f752-8bf2-42bc-9755-c037b83f976e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:27.973430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f752-8bf2-42bc-9755-c037b83f976e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.018095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-1ee0f752-8bf2-42bc-9755-c037b83f976e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:28.018095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1ee0f752-8bf2-42bc-9755-c037b83f976e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.026260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57a245-f32b-447d-ac56-c2fc69211bf9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.079089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.079949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.080288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.080560 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.082437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:28.082743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.084395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.094192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 398/787] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:28.098148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 390/788] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:28.103811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f752-8bf2-42bc-9755-c037b83f976e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.195252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.214888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f752-8bf2-42bc-9755-c037b83f976e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.259090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.259740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.260016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.260287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.268099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.269579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:28.286827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:28.287789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:28.288733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 399/789] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:28.293650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.353893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.377254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.431924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98ea1e-7733-4715-9a2e-7971f2798fad None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.470924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.472138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.473452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.474235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.475190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:28.475785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.478730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.499372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 391/790] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:28.505502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.574661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd50c55-8203-4019-b9ac-695d0390e504 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.611031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeefd52-72db-4db5-81f3-5228e7e1ec1f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.613741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.617330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cc1cb-4ca4-4967-a301-9adc5ab0ab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.617965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cc1cb-4ca4-4967-a301-9adc5ab0ab32 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.618244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cc1cb-4ca4-4967-a301-9adc5ab0ab32 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.618548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cc1cb-4ca4-4967-a301-9adc5ab0ab32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.629436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.630474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cc1cb-4ca4-4967-a301-9adc5ab0ab32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-5f2cc1cb-4ca4-4967-a301-9adc5ab0ab32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:28.636657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:28.637800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:28.638901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:28.639754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 400/791] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:28.640988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035907-d885-416f-92ea-7a911f5413d7 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.641336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035907-d885-416f-92ea-7a911f5413d7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.641676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00035907-d885-416f-92ea-7a911f5413d7 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.647463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 401/792] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:28.648338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00035907-d885-416f-92ea-7a911f5413d7 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.661639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4126807-7f3b-4fee-8286-fed3242efe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.662433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126807-7f3b-4fee-8286-fed3242efe01 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.662774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126807-7f3b-4fee-8286-fed3242efe01 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.663110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126807-7f3b-4fee-8286-fed3242efe01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.678454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4126807-7f3b-4fee-8286-fed3242efe01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.680817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4126807-7f3b-4fee-8286-fed3242efe01 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-d4126807-7f3b-4fee-8286-fed3242efe01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:28.701237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:28.703387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:28.705567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:28.705567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:28.705567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:28.705567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 392/793] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:28.706289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0755a0-c192-4339-aeeb-773dedcd764f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.707292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0755a0-c192-4339-aeeb-773dedcd764f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.707647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0755a0-c192-4339-aeeb-773dedcd764f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.733892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ee0755a0-c192-4339-aeeb-773dedcd764f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:28.736077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ee0755a0-c192-4339-aeeb-773dedcd764f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:28.778571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00035907-d885-416f-92ea-7a911f5413d7 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:28.819899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:28.820531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:28.820858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:28.821265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:28.822028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:28.822429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:28.823896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:28.827838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 393/794] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:28.828876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0755a0-c192-4339-aeeb-773dedcd764f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.841516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 402/795] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:28.846829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:28.957173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.965842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:28.984002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0755a0-c192-4339-aeeb-773dedcd764f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.023370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.024395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.024873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.025394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.026231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:29.027099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.029128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.044094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 394/796] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:29.049217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.057085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee9888-8bba-48ae-93c8-f2b0e90e1b42 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.134520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.135536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.136678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.137163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.163986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.165430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:29.184427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:29.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:29.186283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:29.186283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:29.186283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:29.188390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 403/797] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:29.189433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.216436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b7c00-b369-4dfa-8a93-b98c47ba5dcb None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.251647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.252129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.252401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.252664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.259492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.260918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.262583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:29.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:29.275377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 395/798] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:29.276266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.313597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b759e2-4ab9-4ecb-9760-2ec2f0dcdb81 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.328301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:29.350271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0459f-4184-4c87-9056-ca2682dd965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.351144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0459f-4184-4c87-9056-ca2682dd965b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.351564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0459f-4184-4c87-9056-ca2682dd965b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.352041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0459f-4184-4c87-9056-ca2682dd965b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.363529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0459f-4184-4c87-9056-ca2682dd965b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-a2a0459f-4184-4c87-9056-ca2682dd965b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:29.368607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:29.369638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:29.370504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:29.373139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 404/799] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:29.374143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0459f-4184-4c87-9056-ca2682dd965b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.392225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b576deee-ac97-4a45-852c-d60b1704d45a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.445030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.445581 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.445928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.446264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.447000 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:29.447366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.448800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.462777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 396/800] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:29.463840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.491919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0459f-4184-4c87-9056-ca2682dd965b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.525918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:29.528159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.528996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.529496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.529964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.530976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:29.541275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.541275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.546863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 405/801] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:29.547963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.583021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f6afea-304b-420e-9eaa-02108f45e46d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.584980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:29.629178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.629957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.630230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.630526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.631796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:29.632101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.633993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.645525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 397/802] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:29.646432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.658820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862b903-8136-4a7d-991b-67ad782e550f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.703658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.704669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.705144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.705605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.719137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.721120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:29.734131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:29.735313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:03:29.736249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:29.739006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 406/803] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:29.740043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.790072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6a199-c21f-48d2-bd84-8c8cfc75579c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.818721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:29.837507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db303dd-548c-4981-9d59-0c7c590ffc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.838168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db303dd-548c-4981-9d59-0c7c590ffc33 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.838471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db303dd-548c-4981-9d59-0c7c590ffc33 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.838740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db303dd-548c-4981-9d59-0c7c590ffc33 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/135696b09c2c4f809c49e0af200f32de/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.841715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db303dd-548c-4981-9d59-0c7c590ffc33 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=135696b09c2c4f809c49e0af200f32de, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.843434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db303dd-548c-4981-9d59-0c7c590ffc33 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.846547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:29.854542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b2547-95c2-4413-93f1-29da4de4cb35 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:29.889098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:29.889941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:29.890289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:29.890686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:29.915527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:29.915917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 398/804] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:29 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/135696b09c2c4f809c49e0af200f32de/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:29.917261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db303dd-548c-4981-9d59-0c7c590ffc33 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:29.917914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:29.940124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:29.941838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:29.947402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:29.947402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:29.947402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:29.947402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 407/805] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:29.955803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.108226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db03160-5727-48d2-940c-939471bbcb80 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.114970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db303dd-548c-4981-9d59-0c7c590ffc33 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.147038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd9463-d6b6-4547-bbd7-c98e6f974182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.147655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9463-d6b6-4547-bbd7-c98e6f974182 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.147930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9463-d6b6-4547-bbd7-c98e6f974182 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.148201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd9463-d6b6-4547-bbd7-c98e6f974182 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.148854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9463-d6b6-4547-bbd7-c98e6f974182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:30.149169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9463-d6b6-4547-bbd7-c98e6f974182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.151120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd9463-d6b6-4547-bbd7-c98e6f974182 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.160890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd5cfa-9094-40c3-8c9c-3b215dd2333a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.162398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd5cfa-9094-40c3-8c9c-3b215dd2333a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.162790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd5cfa-9094-40c3-8c9c-3b215dd2333a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.163137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd5cfa-9094-40c3-8c9c-3b215dd2333a None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.163969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd5cfa-9094-40c3-8c9c-3b215dd2333a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:30.164386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd5cfa-9094-40c3-8c9c-3b215dd2333a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.165947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd5cfa-9094-40c3-8c9c-3b215dd2333a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.166420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 408/806] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:30.167960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c9235-30c6-4556-beeb-e49482ce8695 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.168393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c9235-30c6-4556-beeb-e49482ce8695 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.168842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c9235-30c6-4556-beeb-e49482ce8695 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.170871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 409/807] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:30.171757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c9235-30c6-4556-beeb-e49482ce8695 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.185368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:30.187970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 399/808] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:03:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:30.189140 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b3b97-1378-4472-ba40-254b4e7d6a97 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.191957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b3b97-1378-4472-ba40-254b4e7d6a97 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.191957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b3b97-1378-4472-ba40-254b4e7d6a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.209922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bb4b3b97-1378-4472-ba40-254b4e7d6a97 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:30.211543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bb4b3b97-1378-4472-ba40-254b4e7d6a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:30.246568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:30.288272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 400/809] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:30.289442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b3b97-1378-4472-ba40-254b4e7d6a97 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.299655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c9235-30c6-4556-beeb-e49482ce8695 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.385593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.386238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.386543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.386816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.398728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.400161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:30.427108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:30.428035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:30.428977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:30.434125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 410/810] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:30.447373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.453401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b3b97-1378-4472-ba40-254b4e7d6a97 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.494060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8d48a-3c69-422d-aeba-58ebcf0bf853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.494745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8d48a-3c69-422d-aeba-58ebcf0bf853 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.495024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8d48a-3c69-422d-aeba-58ebcf0bf853 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.495591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8d48a-3c69-422d-aeba-58ebcf0bf853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.536716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:30.547781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:30.571523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8d48a-3c69-422d-aeba-58ebcf0bf853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.572980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a8d48a-3c69-422d-aeba-58ebcf0bf853 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.600212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c5091-22c8-4775-8b2e-1f6ab01b123a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.649749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b942fd8-38c4-4200-abe1-407bad592c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.650642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b942fd8-38c4-4200-abe1-407bad592c94 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.651099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b942fd8-38c4-4200-abe1-407bad592c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.651512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b942fd8-38c4-4200-abe1-407bad592c94 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.652389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b942fd8-38c4-4200-abe1-407bad592c94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:30.653387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b942fd8-38c4-4200-abe1-407bad592c94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.654990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b942fd8-38c4-4200-abe1-407bad592c94 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.679771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 411/811] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:30.681434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e939cc-c532-427d-9ffd-e2f303556270 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.681920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e939cc-c532-427d-9ffd-e2f303556270 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.682434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e939cc-c532-427d-9ffd-e2f303556270 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.683869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 412/812] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:30.684857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e939cc-c532-427d-9ffd-e2f303556270 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.693633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 401/813] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:03:30.695284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8d48a-3c69-422d-aeba-58ebcf0bf853 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.767695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:30.801593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e939cc-c532-427d-9ffd-e2f303556270 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.803001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a8d48a-3c69-422d-aeba-58ebcf0bf853 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:30.828804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:30.839309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.839950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.840248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.840517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.842531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984044e7-6dc1-4188-b986-dc475ed3e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:30.842531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984044e7-6dc1-4188-b986-dc475ed3e6cc None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.842531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984044e7-6dc1-4188-b986-dc475ed3e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.842531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984044e7-6dc1-4188-b986-dc475ed3e6cc None admin] PATH_INFO: `/v3/services/network` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.842857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984044e7-6dc1-4188-b986-dc475ed3e6cc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.844483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984044e7-6dc1-4188-b986-dc475ed3e6cc None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.852181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:30.853653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-984044e7-6dc1-4188-b986-dc475ed3e6cc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:30.853875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:30.855339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:30.856179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 413/814] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:30.856968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5697eff-489b-41e1-affb-725ff35f50f8 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:30.856968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5697eff-489b-41e1-affb-725ff35f50f8 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:30.856968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5697eff-489b-41e1-affb-725ff35f50f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:30.875908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:30.876881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:30.877792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:30.877792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:30.877792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:30.877961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a5697eff-489b-41e1-affb-725ff35f50f8 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:30.878043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 402/815] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:30.879295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:30.882512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a5697eff-489b-41e1-affb-725ff35f50f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:30.969431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 414/816] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:30.970288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5697eff-489b-41e1-affb-725ff35f50f8 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.008854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f4f8b-83b5-4d7a-9df7-40ecbc3b8193 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.033881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:31.053469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.055009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.055466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.056149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.070417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.073085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:31.092476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:31.093440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:31.094299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:31.094299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:31.094299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:31.094518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5697eff-489b-41e1-affb-725ff35f50f8 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.101562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 403/817] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:31.101562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.117320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:31.137429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.138079 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.138425 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.139054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.140253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] RBAC: Adding query filter params (name=network) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:31.141245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.143360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.154363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 415/818] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:31.154363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.225817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8588f61-56c1-4d85-9635-b2f43c93d801 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.262634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.263529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.263865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.264478 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.265430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:31.265735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.267584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.273228 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9122e-700a-4466-af4a-6a771dd8ffaa None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.283752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 404/819] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:31.283752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.319149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:31.321917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.322560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.322839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.323745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.323745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:31.324032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.325452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.338253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 416/820] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:31.339084 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.388264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:31.395890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d4f96-05c4-4ffa-9f15-1364f7aac79b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.437941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.438578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.438852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.439121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.439744 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:31.440068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.442002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.451583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 405/821] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:31.451583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.457790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d1d09-ceaa-4c7b-94fc-6545e5718d3f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.493181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.493725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.494136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.494244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.494841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.496164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:31.504897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:31.505868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:31.507094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 417/822] 10.222.0.20 () {56 vars in 1129 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:31.507963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.551758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:31.557277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af20be-9faf-4359-ba45-a03934a4d4f2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.591637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.592253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.592527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.592794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.599750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.601179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:31.612128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:31.613585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:31.615021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:31.615021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:31.615021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:31.615021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:31.615021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:31.615021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:31.615021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:31.615021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:31.615474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791a510-e43b-4d3e-b830-f4edc9558133 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.615550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 406/823] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:31.615550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.628057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:31.657212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.657945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.658215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.658509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.665485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.666942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:31.678878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:31.679777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 418/824] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:31.680688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.726066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271093b-1b0a-473f-a98c-7cf3c63e7fb1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.763227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5450a-0ddd-4a5c-9471-7b276e97494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.763783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5450a-0ddd-4a5c-9471-7b276e97494b None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.764028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5450a-0ddd-4a5c-9471-7b276e97494b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.764482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5450a-0ddd-4a5c-9471-7b276e97494b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.765146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5450a-0ddd-4a5c-9471-7b276e97494b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.766566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5450a-0ddd-4a5c-9471-7b276e97494b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.791152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc787bd-6397-469d-8b87-961b2b4db073 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:31.826204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:31.826834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:31.827156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:31.827426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:31.828041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:31.828357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:31.829679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:31.834884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 419/825] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:31.835719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.849481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 407/826] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:31 2022] POST /identity/v3/endpoints => generated 337 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:31.850653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5450a-0ddd-4a5c-9471-7b276e97494b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:31.867565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:31.885452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:31.983937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcb8b4-e21b-45cd-abfd-8046fb43faa4 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:32.000249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5450a-0ddd-4a5c-9471-7b276e97494b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:32.025517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:32.026098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.026404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.026677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.027375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:32.027676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:32.029413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:32.047452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 420/827] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:32.047452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:32.060169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:32.061301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.061732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.062142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.063237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:32.063955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:32.065396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:32.082430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 408/828] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:32.082430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:32.165212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:32.173480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:32.184312 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186303f-9780-4e09-a2d9-2b3c01db68b7 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:32.200200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd0681-9232-40bb-84ca-566aea84f357 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:32.224458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:32.225061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.225338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.225605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.226225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:32.226593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:32.227945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:32.233289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 421/829] 10.222.0.20 () {56 vars in 1124 bytes} [Fri Sep 9 23:03:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:32.234373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:32.236769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3da0f-12d9-4984-8649-44a06a748781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:32.238330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3da0f-12d9-4984-8649-44a06a748781 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.238632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3da0f-12d9-4984-8649-44a06a748781 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.238906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3da0f-12d9-4984-8649-44a06a748781 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/2c8bfd34442b48a78600f568b534eaa1/roles/8dc6b765036246cfaf05b6553832c86a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.254392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3da0f-12d9-4984-8649-44a06a748781 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=2c8bfd34442b48a78600f568b534eaa1, role_id=8dc6b765036246cfaf05b6553832c86a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:32.255813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3da0f-12d9-4984-8649-44a06a748781 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:32.327224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 409/830] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:32 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/2c8bfd34442b48a78600f568b534eaa1/roles/8dc6b765036246cfaf05b6553832c86a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:32.328027 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3da0f-12d9-4984-8649-44a06a748781 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:32.333940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb00a8b-7f26-4418-b98e-a57a06633350 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:32.374872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc9026-52e3-40b5-abd6-0239f799a7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:32.375482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc9026-52e3-40b5-abd6-0239f799a7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.375754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc9026-52e3-40b5-abd6-0239f799a7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.376015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc9026-52e3-40b5-abd6-0239f799a7f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.376847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc9026-52e3-40b5-abd6-0239f799a7f4 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c3bec982c8474c349ca4901039844d26) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:32.377264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc9026-52e3-40b5-abd6-0239f799a7f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:32.378816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc9026-52e3-40b5-abd6-0239f799a7f4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:32.387674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 422/831] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:32 2022] GET /identity/v3/role_assignments?user.id=c3bec982c8474c349ca4901039844d26&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:32.388992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b020a-b743-424c-ad18-c7568e98b56f None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.389270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b020a-b743-424c-ad18-c7568e98b56f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.389535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b020a-b743-424c-ad18-c7568e98b56f None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.391133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 423/832] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:32.397540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f574fa-d559-4f79-9187-892b42c997ce None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.397819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f574fa-d559-4f79-9187-892b42c997ce None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.398093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f574fa-d559-4f79-9187-892b42c997ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.432584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c6f574fa-d559-4f79-9187-892b42c997ce None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:32.434043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c6f574fa-d559-4f79-9187-892b42c997ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:32.459103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:32.490427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3da0f-12d9-4984-8649-44a06a748781 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:32.518166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:32.535670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 424/833] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:32.539973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b5d5-5814-4b19-bc64-4b57ff7cd6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:32.540596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b5d5-5814-4b19-bc64-4b57ff7cd6e1 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.540869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b5d5-5814-4b19-bc64-4b57ff7cd6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.541134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b5d5-5814-4b19-bc64-4b57ff7cd6e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.541814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b5d5-5814-4b19-bc64-4b57ff7cd6e1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:32.543322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b5d5-5814-4b19-bc64-4b57ff7cd6e1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:32.609174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 410/834] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:32.816617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f574fa-d559-4f79-9187-892b42c997ce None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:32.921577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f574fa-d559-4f79-9187-892b42c997ce None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:32.962654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e73c2-85ad-4278-b56a-aed8e957a8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:32.963327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e73c2-85ad-4278-b56a-aed8e957a8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:32.963603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e73c2-85ad-4278-b56a-aed8e957a8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:32.963868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e73c2-85ad-4278-b56a-aed8e957a8c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:32.974804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e73c2-85ad-4278-b56a-aed8e957a8c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-cc8e73c2-85ad-4278-b56a-aed8e957a8c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:32.979568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:32.980467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:32.981295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 425/835] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:32.986723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b5d5-5814-4b19-bc64-4b57ff7cd6e1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:33.092214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b5d5-5814-4b19-bc64-4b57ff7cd6e1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:33.127455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:33.128059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.128332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.128629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:33.129280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:33.129577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:33.130966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:33.139968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 411/836] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:33.146444 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e73c2-85ad-4278-b56a-aed8e957a8c8 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:33.151306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:33.255200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e73c2-85ad-4278-b56a-aed8e957a8c8 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:33.294101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:33.294749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.295022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.295288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:33.307311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:33.308309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:33.324380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:33.325544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:33.326826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:33.326826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:33.326826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:33.326826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 426/837] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:33.331384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:33.368391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:33.443420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0913a-933c-401a-9e38-0554c2a8fe44 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:33.479207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:33.480176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.480664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.481146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:33.481981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:33.482509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:33.484062 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:33.496475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 412/838] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:33.504809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:33.559602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:33.607762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a917be-5d64-4494-8ac9-b2ff672aed3a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:33.647945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:33.648567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.648840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.649108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:33.657029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:33.658474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:33.670560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:33.671488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:33.672426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 427/839] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:03:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:33.677809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:33.776322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:33.782369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e029f-8f7f-4715-b05d-db5579ddf480 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:33.818918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:33.819594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.819877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.820235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:33.820972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:33.821274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:33.822725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:33.841379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 413/840] 10.222.0.20 () {56 vars in 1135 bytes} [Fri Sep 9 23:03:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:33.846705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:33.954981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5b87e-550c-40c1-bea6-072efb918a42 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:33.974088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:33.992077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b697a2-2fa9-4363-b23a-2e643d572e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:33.992722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b697a2-2fa9-4363-b23a-2e643d572e54 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:33.993178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b697a2-2fa9-4363-b23a-2e643d572e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:33.993451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b697a2-2fa9-4363-b23a-2e643d572e54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.063112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b697a2-2fa9-4363-b23a-2e643d572e54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:34.064544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b697a2-2fa9-4363-b23a-2e643d572e54 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:34.167623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 428/841] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:03:34.175798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:34.278808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e303cc5-8bcc-46f6-9a7f-2f4cf771ea30 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:34.314458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:34.315489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.315984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.317002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.327880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:34.329608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:34.345791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:34.346787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:34.348398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:34.348398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:34.348398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:34.350534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 414/842] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:34.356321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b697a2-2fa9-4363-b23a-2e643d572e54 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:34.374570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:34.464675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b697a2-2fa9-4363-b23a-2e643d572e54 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:34.499972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad57898-b314-4131-85cb-f92f24a04ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:34.500570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad57898-b314-4131-85cb-f92f24a04ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.500841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad57898-b314-4131-85cb-f92f24a04ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.501106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad57898-b314-4131-85cb-f92f24a04ef5 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.501731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad57898-b314-4131-85cb-f92f24a04ef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:34.502025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad57898-b314-4131-85cb-f92f24a04ef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:34.503453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad57898-b314-4131-85cb-f92f24a04ef5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:34.515602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 429/843] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:34.521464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:34.531436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:34.637106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0d55a-86c5-481a-8322-5137124e0c4a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:34.658398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368f1ae-f8ca-460a-94b3-2134c0d99686 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.658712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368f1ae-f8ca-460a-94b3-2134c0d99686 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.658954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368f1ae-f8ca-460a-94b3-2134c0d99686 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.660754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 430/844] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:34.666420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39b2c2-f441-4d3d-bd80-cb7ec4788130 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.666704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39b2c2-f441-4d3d-bd80-cb7ec4788130 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.666978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39b2c2-f441-4d3d-bd80-cb7ec4788130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.677944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:34.679362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.679895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.680371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.687426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-fe39b2c2-f441-4d3d-bd80-cb7ec4788130 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:34.688658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fe39b2c2-f441-4d3d-bd80-cb7ec4788130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:34.689484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:34.691413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:34.704978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:34.705944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:34.707971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:34.707971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:34.707971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:34.707971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:34.707971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:34.707971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:34.707971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:34.707971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:34.710541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 415/845] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:34.716655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:34.767457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 431/846] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:34.795549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f99ca-37dc-4620-ab9f-3a807c878ff6 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.795864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f99ca-37dc-4620-ab9f-3a807c878ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.796098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f99ca-37dc-4620-ab9f-3a807c878ff6 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.797576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 432/847] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:34.801870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:34.802768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc0b1-b354-467e-b4cc-250b8d064b25 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.802994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc0b1-b354-467e-b4cc-250b8d064b25 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.803304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbc0b1-b354-467e-b4cc-250b8d064b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.819358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:34.826773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-dabbc0b1-b354-467e-b4cc-250b8d064b25 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:34.828348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-dabbc0b1-b354-467e-b4cc-250b8d064b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:34.845118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011eaf-ebaf-4589-984f-aa6202af05de None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:34.886240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:34.887266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.887770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.888306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.889285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:34.889840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:34.891798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:34.911417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 416/848] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:34.913452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:34.918919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 433/849] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:34.992422 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018359-1cbc-4330-b70a-de8d8c891f38 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.993011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018359-1cbc-4330-b70a-de8d8c891f38 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:34.993011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56018359-1cbc-4330-b70a-de8d8c891f38 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:34.994492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 434/850] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:34.999614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad35686-9a24-43c7-8e81-cdf5bdf00c1f None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:34.999889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad35686-9a24-43c7-8e81-cdf5bdf00c1f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.000160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad35686-9a24-43c7-8e81-cdf5bdf00c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.018286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-cad35686-9a24-43c7-8e81-cdf5bdf00c1f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:35.019681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cad35686-9a24-43c7-8e81-cdf5bdf00c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:35.027755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d962a-5c62-4977-9be4-fd3f84d73777 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.058965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.067969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.068571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.071350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.071350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.071350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=135696b09c2c4f809c49e0af200f32de) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:35.071350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.071636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.077911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 417/851] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:34 2022] GET /identity/v3/role_assignments?user.id=135696b09c2c4f809c49e0af200f32de&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:35.079334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.098173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 435/852] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:35.204761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec8c62-cbcd-43fb-9f73-d18840c99a63 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.211307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad35686-9a24-43c7-8e81-cdf5bdf00c1f None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.246554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c529-1c49-4da3-a65a-26439c1126e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.247228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c529-1c49-4da3-a65a-26439c1126e4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.247508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c529-1c49-4da3-a65a-26439c1126e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.247896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4c529-1c49-4da3-a65a-26439c1126e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.260168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4c529-1c49-4da3-a65a-26439c1126e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5dc4c529-1c49-4da3-a65a-26439c1126e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:35.266205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:35.267229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:35.268154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 418/853] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:35.275444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c529-1c49-4da3-a65a-26439c1126e4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.287352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.333603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad35686-9a24-43c7-8e81-cdf5bdf00c1f None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.340613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.378297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.379274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.379600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.379875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.380458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.381819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:35.394865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:35.396037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:35.396902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 436/854] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:35.397688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.412785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4c529-1c49-4da3-a65a-26439c1126e4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.452175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde315be-a6bd-4284-8835-907898492598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.452786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde315be-a6bd-4284-8835-907898492598 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.453066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde315be-a6bd-4284-8835-907898492598 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.453339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde315be-a6bd-4284-8835-907898492598 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.453997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde315be-a6bd-4284-8835-907898492598 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:35.454779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde315be-a6bd-4284-8835-907898492598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.456427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde315be-a6bd-4284-8835-907898492598 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.470620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 419/855] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:35.470620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde315be-a6bd-4284-8835-907898492598 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.503703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b3239-c583-4b89-a14d-2c5f22636117 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.534146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.541092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.541717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.542096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.542254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.548978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.548978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:35.552834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:35.553853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:35.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:35.558504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 437/856] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:35.559582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.580855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde315be-a6bd-4284-8835-907898492598 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.596453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.616348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.616960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.617233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.617549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.618127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:35.618448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.619819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.626010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 420/857] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:35.627233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.681221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262775ad-089e-40be-9bc8-b8bccb74c507 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.721840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.722639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.723508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.723780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.734611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32161118-ee13-4ce2-9e63-15274d27afe0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.736415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.738872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:35.757621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:35.758612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:35.759505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:35.759505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:35.759505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:35.759505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 438/858] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:35.759737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.775424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.776025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.776299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.776570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.777190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:35.777484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.778872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.788057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 421/859] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:35.788818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.808298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.830602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:35.873472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e169d28-5af4-41b7-aa98-1383d97343ad None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.892827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a23d5-fb6f-46cd-858d-8ff1616713f6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:35.911972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.912580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.912858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.913131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.913761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:35.914065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.915486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.927524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 439/860] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:35.927524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:35.941708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:35.942813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:35.943198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:35.943567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:35.944953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:35.945373 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:35.946850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:35.969185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 422/861] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:35.974992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.063701 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab7016-9045-4f2a-9487-cb8be8aa4e21 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.080352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:36.109869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:36.115128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52b23-97ab-4365-89ea-e8d0f9f4d693 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.124445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.125057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.125327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.125596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.126220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:36.126653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.128318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.135088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 440/862] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:36.163714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.163714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.163714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.163714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.170711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.172419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:36.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:36.187472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:36.188408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 423/863] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:03:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:36.195513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.316397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6c7a1-f030-4bbd-9f78-a157e2d59d53 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.355260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.355859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.356129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.356391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.357109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:36.357405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.358839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.378278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 441/864] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Sep 9 23:03:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:36.384156 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.495611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2aab4-f2fa-4c81-9d9c-e4ef256eabff None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.532184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e723501-97ab-4234-9e93-95c0fb88bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.533192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e723501-97ab-4234-9e93-95c0fb88bf36 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.533671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e723501-97ab-4234-9e93-95c0fb88bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.534139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e723501-97ab-4234-9e93-95c0fb88bf36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.564103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:36.607434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e723501-97ab-4234-9e93-95c0fb88bf36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.609204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e723501-97ab-4234-9e93-95c0fb88bf36 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.718422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 424/865] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:03:36.724316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.759663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:36.838727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49679aae-512a-42c0-b63d-9662562a2d54 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:36.875571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:36.876114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:36.876394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:36.876714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:36.888331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:36.889779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:36.905763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:36.906776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:36.907909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:36.907909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:36.907909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:36.907909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 442/866] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:36.913177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e723501-97ab-4234-9e93-95c0fb88bf36 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:36.979575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:37.028987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e723501-97ab-4234-9e93-95c0fb88bf36 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:37.071310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:37.071924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:37.072201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:37.072470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:37.073177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:37.073477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:37.074876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:37.088437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 425/867] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:37.093800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:37.184573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:37.200616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efa0c-ff09-4883-823a-40d5cc4abae0 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:37.236399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:37.237016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:37.237286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:37.237549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:37.244513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:37.245935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:37.257428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:37.258317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:37.259260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 443/868] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:37.265987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:37.377723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380ab3a-8ebe-4d21-9a93-235b64fc9d3f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:37.399291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:37.414246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13706bec-56d0-426b-8169-04a5226c851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:37.414882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13706bec-56d0-426b-8169-04a5226c851c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:37.415160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13706bec-56d0-426b-8169-04a5226c851c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:37.415428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13706bec-56d0-426b-8169-04a5226c851c None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:37.416120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13706bec-56d0-426b-8169-04a5226c851c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:37.416420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13706bec-56d0-426b-8169-04a5226c851c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:37.417834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13706bec-56d0-426b-8169-04a5226c851c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:37.430381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 426/869] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:37.437106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:37.479723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602397b-66ef-4ee7-855d-154d70edb73d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:37.480054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602397b-66ef-4ee7-855d-154d70edb73d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:37.480268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602397b-66ef-4ee7-855d-154d70edb73d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:37.481859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 427/870] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:37.486842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2344-973f-44d2-a3b8-27a19832691e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:37.487126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2344-973f-44d2-a3b8-27a19832691e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:37.487402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f2344-973f-44d2-a3b8-27a19832691e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:37.505522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-210f2344-973f-44d2-a3b8-27a19832691e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:37.506746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-210f2344-973f-44d2-a3b8-27a19832691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:37.550254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f529a08-4e6a-4adf-9be8-4149ef02840b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:37.586411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 428/871] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:37.591594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:37.592474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:37.592985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:37.593454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:37.594452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=2c8bfd34442b48a78600f568b534eaa1) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:37.594961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:37.596638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:37.603182 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 444/872] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:37 2022] GET /identity/v3/role_assignments?user.id=2c8bfd34442b48a78600f568b534eaa1&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:37.854752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f2344-973f-44d2-a3b8-27a19832691e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:37.962023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f2344-973f-44d2-a3b8-27a19832691e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:37.975231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:38.012643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:38.013457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:38.013739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:38.014738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:38.015341 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:38.016720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:38.027861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:38.028858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:38.029714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 429/873] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:38.037600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:38.053368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:38.144766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1042715-c19d-4e0c-976f-af884f6ca3ef None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:38.180128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7acf7d-a8ee-4872-b5c4-2249208f1516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:38.180795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7acf7d-a8ee-4872-b5c4-2249208f1516 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:38.181072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7acf7d-a8ee-4872-b5c4-2249208f1516 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:38.181340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7acf7d-a8ee-4872-b5c4-2249208f1516 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:38.181980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7acf7d-a8ee-4872-b5c4-2249208f1516 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:38.182274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7acf7d-a8ee-4872-b5c4-2249208f1516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:38.183711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7acf7d-a8ee-4872-b5c4-2249208f1516 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:38.189522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 445/874] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:38.195877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:38.308339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b8bf-dac4-45a8-bafc-e9b26c3877cd None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:38.308673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b8bf-dac4-45a8-bafc-e9b26c3877cd None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:38.308888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b8bf-dac4-45a8-bafc-e9b26c3877cd None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:38.310362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 446/875] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:38.314403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f6eab-1349-421e-8efd-4942a8a441cc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:38.317593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08e19-75fa-4c1d-bec3-d730d5d71912 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:38.317879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08e19-75fa-4c1d-bec3-d730d5d71912 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:38.318148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c08e19-75fa-4c1d-bec3-d730d5d71912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:38.341154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c8c08e19-75fa-4c1d-bec3-d730d5d71912 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:38.342381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c8c08e19-75fa-4c1d-bec3-d730d5d71912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:38.359612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104e35c-46de-414b-807c-93d5e470a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:38.360346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104e35c-46de-414b-807c-93d5e470a87d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:38.360627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104e35c-46de-414b-807c-93d5e470a87d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:38.360999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104e35c-46de-414b-807c-93d5e470a87d None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:38.361672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104e35c-46de-414b-807c-93d5e470a87d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:38.361975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104e35c-46de-414b-807c-93d5e470a87d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:38.364501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104e35c-46de-414b-807c-93d5e470a87d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:38.370017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 430/876] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:38.435089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 447/877] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:38.452588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eee9d-d0a1-4a7d-ac47-5b24703fe9a9 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:38.452868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eee9d-d0a1-4a7d-ac47-5b24703fe9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:38.453139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eee9d-d0a1-4a7d-ac47-5b24703fe9a9 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:38.454861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 431/878] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:38.463297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0bdd3-65d6-44eb-ab39-f8cad054ae2e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:38.463750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0bdd3-65d6-44eb-ab39-f8cad054ae2e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:38.464140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0bdd3-65d6-44eb-ab39-f8cad054ae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:38.471128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:38.484669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6df0bdd3-65d6-44eb-ab39-f8cad054ae2e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:38.485894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6df0bdd3-65d6-44eb-ab39-f8cad054ae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:38.570201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 448/879] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:38.745510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eee9d-d0a1-4a7d-ac47-5b24703fe9a9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:38.788773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:38.851048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eee9d-d0a1-4a7d-ac47-5b24703fe9a9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:38.857509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0bdd3-65d6-44eb-ab39-f8cad054ae2e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:38.891245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1ad5c-fa5e-4c77-8954-c1337812066d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:38.892483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1ad5c-fa5e-4c77-8954-c1337812066d None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:38.893330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1ad5c-fa5e-4c77-8954-c1337812066d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:38.894039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1ad5c-fa5e-4c77-8954-c1337812066d None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:38.895522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1ad5c-fa5e-4c77-8954-c1337812066d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:38.897400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1ad5c-fa5e-4c77-8954-c1337812066d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:38.972026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0bdd3-65d6-44eb-ab39-f8cad054ae2e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:38.994570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 432/880] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:03:38 2022] POST /identity/v3/services => generated 269 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:39.013678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:39.030268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65017bfd-e44e-4776-8770-e1d5a74c3316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:39.031233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65017bfd-e44e-4776-8770-e1d5a74c3316 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:39.031725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65017bfd-e44e-4776-8770-e1d5a74c3316 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:39.032175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65017bfd-e44e-4776-8770-e1d5a74c3316 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:39.033031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65017bfd-e44e-4776-8770-e1d5a74c3316 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:39.034630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65017bfd-e44e-4776-8770-e1d5a74c3316 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:39.130546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 449/881] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:03:38 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:39.953719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dabbd-daaf-4ade-ba4f-69b4961ced54 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:39.954001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dabbd-daaf-4ade-ba4f-69b4961ced54 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:39.954463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474dabbd-daaf-4ade-ba4f-69b4961ced54 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:39.956430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 433/882] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:39.962475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703d24-f5bf-4671-a4b9-0cacd3e0f245 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:39.963330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703d24-f5bf-4671-a4b9-0cacd3e0f245 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:39.963811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d703d24-f5bf-4671-a4b9-0cacd3e0f245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:39.981969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4d703d24-f5bf-4671-a4b9-0cacd3e0f245 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:39.983447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4d703d24-f5bf-4671-a4b9-0cacd3e0f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:40.064966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 450/883] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:40.333311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dabbd-daaf-4ade-ba4f-69b4961ced54 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.444313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:40.449218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474dabbd-daaf-4ade-ba4f-69b4961ced54 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:40.484375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959412-b487-4ff4-900b-e064fadb1f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:40.485062 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959412-b487-4ff4-900b-e064fadb1f48 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.485343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959412-b487-4ff4-900b-e064fadb1f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.485614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3959412-b487-4ff4-900b-e064fadb1f48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.496554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3959412-b487-4ff4-900b-e064fadb1f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b3959412-b487-4ff4-900b-e064fadb1f48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:40.501315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:40.502360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:40.503310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 434/884] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Sep 9 23:03:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:40.510205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703d24-f5bf-4671-a4b9-0cacd3e0f245 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.620426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d703d24-f5bf-4671-a4b9-0cacd3e0f245 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:40.638001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:40.659598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9fd0a-eaa4-4b51-b0c0-845339d976df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:40.660659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9fd0a-eaa4-4b51-b0c0-845339d976df None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.661156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9fd0a-eaa4-4b51-b0c0-845339d976df None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.661641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9fd0a-eaa4-4b51-b0c0-845339d976df None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.662540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9fd0a-eaa4-4b51-b0c0-845339d976df None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:40.663062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9fd0a-eaa4-4b51-b0c0-845339d976df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:40.664680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9fd0a-eaa4-4b51-b0c0-845339d976df None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:40.673452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 451/885] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Sep 9 23:03:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:40.679533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959412-b487-4ff4-900b-e064fadb1f48 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.737803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5aa97-a7c2-4a1b-b0cb-5a5b9aaf16b4 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.739028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5aa97-a7c2-4a1b-b0cb-5a5b9aaf16b4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.739531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5aa97-a7c2-4a1b-b0cb-5a5b9aaf16b4 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.741310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 452/886] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:40.746856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611154-a71b-486d-b2bc-0f49de4293dd None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.747337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611154-a71b-486d-b2bc-0f49de4293dd None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.748434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21611154-a71b-486d-b2bc-0f49de4293dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.767920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-21611154-a71b-486d-b2bc-0f49de4293dd None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:40.770147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-21611154-a71b-486d-b2bc-0f49de4293dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:40.785551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3959412-b487-4ff4-900b-e064fadb1f48 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:40.822543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:40.823132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:40.823409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:40.823677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:40.835819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:40.837285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:40.853793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:40.854951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:40.855884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:40.855884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:40.855884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:40.855884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 435/888] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:40.856837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 453/888] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:40.860635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:40.873581 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:40.971057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac47b1-c389-454f-a94c-d2f53ce3c202 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.015715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.016751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.017232 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.017740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.018777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:41.019401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.021024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.035258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 436/889] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:41.043027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21611154-a71b-486d-b2bc-0f49de4293dd None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.145969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21611154-a71b-486d-b2bc-0f49de4293dd None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.153859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:41.177600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.185791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.186450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.186721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.186989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.198018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.200099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:41.217643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:41.218887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:41.219869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 454/890] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:41.225529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.260811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:41.294086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3aa34e-4b4f-47a7-adcf-eeee8a75aad9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.335480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378e7ce-542c-4fa2-9be8-3cba18ad79d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.336162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e7ce-542c-4fa2-9be8-3cba18ad79d3 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.336391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e7ce-542c-4fa2-9be8-3cba18ad79d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.336490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551afd0-445d-4452-9970-d415a98b7ba9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.336660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378e7ce-542c-4fa2-9be8-3cba18ad79d3 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.337256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378e7ce-542c-4fa2-9be8-3cba18ad79d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.338645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b378e7ce-542c-4fa2-9be8-3cba18ad79d3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.372648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c884a8-c4fc-4a54-981d-b56a99eaf9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.373251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c884a8-c4fc-4a54-981d-b56a99eaf9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.373526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c884a8-c4fc-4a54-981d-b56a99eaf9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.373791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c884a8-c4fc-4a54-981d-b56a99eaf9a5 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.374539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c884a8-c4fc-4a54-981d-b56a99eaf9a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:41.374839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c884a8-c4fc-4a54-981d-b56a99eaf9a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.376250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c884a8-c4fc-4a54-981d-b56a99eaf9a5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.398873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 455/891] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:41.400046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752073c7-ff8e-4e37-8f98-459a799a982c None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.400331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752073c7-ff8e-4e37-8f98-459a799a982c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.400599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752073c7-ff8e-4e37-8f98-459a799a982c None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.401947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 456/892] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:41.406533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752073c7-ff8e-4e37-8f98-459a799a982c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.415951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 437/893] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:03:41 2022] POST /identity/v3/services => generated 241 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:41.417432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee634-8e87-4015-a0d0-f3d580b706d7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.417785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee634-8e87-4015-a0d0-f3d580b706d7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.418064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee634-8e87-4015-a0d0-f3d580b706d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.438262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d94ee634-8e87-4015-a0d0-f3d580b706d7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:41.439540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d94ee634-8e87-4015-a0d0-f3d580b706d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:41.440854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:41.526381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 438/894] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:41.526381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12f6d8-637b-4f07-8567-bd7c12ceaf95 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.526381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12f6d8-637b-4f07-8567-bd7c12ceaf95 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.526381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12f6d8-637b-4f07-8567-bd7c12ceaf95 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.526381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 439/895] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:41.530374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d23b-fe18-478a-a247-484d990e0d54 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.530374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d23b-fe18-478a-a247-484d990e0d54 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.530596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033d23b-fe18-478a-a247-484d990e0d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.546962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752073c7-ff8e-4e37-8f98-459a799a982c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.551123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a033d23b-fe18-478a-a247-484d990e0d54 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:41.552345 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a033d23b-fe18-478a-a247-484d990e0d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:41.594543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0f911-f826-4f43-863e-fdb521f69b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.595154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f911-f826-4f43-863e-fdb521f69b69 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.595431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f911-f826-4f43-863e-fdb521f69b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.595703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f911-f826-4f43-863e-fdb521f69b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.613024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:41.634377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 440/896] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:41.685218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0f911-f826-4f43-863e-fdb521f69b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.686698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0f911-f826-4f43-863e-fdb521f69b69 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.713078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:41.796378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 457/897] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:03:41.802238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033d23b-fe18-478a-a247-484d990e0d54 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.820971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0f911-f826-4f43-863e-fdb521f69b69 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:41.931413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0f911-f826-4f43-863e-fdb521f69b69 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.932174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033d23b-fe18-478a-a247-484d990e0d54 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:41.957399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:41.969256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.970125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.971488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:41.971625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.971625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.971781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:41.971781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:41.971906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.971982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:41.972928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.982561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:41.983743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:41.984710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:41.986369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:41.987157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:41.987235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 458/898] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:41.991344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:42.005049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:42.006139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:42.011615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:42.011615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:42.011615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:42.011615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 441/899] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:42.011615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.053529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:42.102197 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e1102-6695-4211-9849-d4ad376c2c14 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.117693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5c1df-bfa9-4c62-8b37-60e69fec51dc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.138993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.139623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.139900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.140169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.140762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.142107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:42.151797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:42.152816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:42.153616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 459/900] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:42.154411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.161865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.162508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.162783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.163049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.163678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.163976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.165337 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.175217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 442/901] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:42.175217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.217653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:42.269767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87425ee-50a7-4341-b4c9-8dee2ecbade7 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.277883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8fc6c2-8501-4cc0-a904-47a7a707264b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.279913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:42.305513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.306117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.306421 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.306904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.307330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.307630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.308979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.315381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.316692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.317118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.317397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.318465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.318820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.320172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.321448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 460/902] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:42.322149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.328030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 443/903] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:42.329077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.435423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82875c-6e96-47f8-a994-74f93eaa7e12 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.449783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:42.452894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81000-c815-429c-82cd-816630dfd1cf None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.474310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.474869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.475148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.475565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.476132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.476686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.478356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.484171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 461/904] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:42.484849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.494189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.494859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.495138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.495407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.502761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.505079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:42.516548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:42.517446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:42.518362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 444/905] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:42.518918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.547641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:42.598651 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21a019-0c81-4fad-981c-73ce36d409e1 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.622449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4fd9-a2e6-4cbc-b6ed-9eb645ffe937 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.636790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.637850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.638126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.638556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.639294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.639592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.641174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.647165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 462/906] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:42.648234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.659781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.660673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.661193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.661676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.662544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] RBAC: Adding query filter params (interface=public, service_id=073595283e0e4f3fb818ffc056b8b187, region_id=RegionOne) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.663017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.664717 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.670112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 445/907] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/endpoints?service_id=073595283e0e4f3fb818ffc056b8b187&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:42.671136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.695303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:42.756896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3fedf4-4c64-4d96-9ec9-7b8ee0eeb1f3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.777198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:42.787012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c922787-1917-44e5-b120-b366d9b11ace None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.792301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.792900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.793173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.793438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.794121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.794442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.795884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.814099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 463/908] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:42.814099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.833752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.834783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.835229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.835624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.836563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] RBAC: Adding query filter params (interface=public, service_id=03c7458d2be04c15b5bdf2f08f2e33cf, region_id=RegionOne) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.837053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.839390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.852019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 446/909] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/endpoints?service_id=03c7458d2be04c15b5bdf2f08f2e33cf&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:42.852019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.927906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:42.931043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6cc40-f7a1-4776-b766-6afc3a3a83df None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.949839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d822c028-4357-428b-94d1-abd689c37a36 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:42.969006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce21ea7-6e53-416b-b2f9-634d7247359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.969602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21ea7-6e53-416b-b2f9-634d7247359c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.969877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21ea7-6e53-416b-b2f9-634d7247359c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.970142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce21ea7-6e53-416b-b2f9-634d7247359c None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.970768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce21ea7-6e53-416b-b2f9-634d7247359c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.972108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce21ea7-6e53-416b-b2f9-634d7247359c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:42.986372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 464/910] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:42.986372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce21ea7-6e53-416b-b2f9-634d7247359c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:42.994279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1aa873-5a5f-4cf8-bd1b-36e537c7814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:42.995090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1aa873-5a5f-4cf8-bd1b-36e537c7814a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:42.995395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1aa873-5a5f-4cf8-bd1b-36e537c7814a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:42.995729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1aa873-5a5f-4cf8-bd1b-36e537c7814a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:42.996535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1aa873-5a5f-4cf8-bd1b-36e537c7814a None admin] RBAC: Adding query filter params (role.id=a65ba9a33bc44775a27c2cfdd3a54a42, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=2c8bfd34442b48a78600f568b534eaa1) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:42.996906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1aa873-5a5f-4cf8-bd1b-36e537c7814a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:42.998474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1aa873-5a5f-4cf8-bd1b-36e537c7814a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:43.005209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 447/911] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/role_assignments?user.id=2c8bfd34442b48a78600f568b534eaa1&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=a65ba9a33bc44775a27c2cfdd3a54a42 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:43.105657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce21ea7-6e53-416b-b2f9-634d7247359c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:43.152183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b91664-fdeb-4c19-9c17-96ca01ba3003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:43.152815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91664-fdeb-4c19-9c17-96ca01ba3003 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:43.153089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91664-fdeb-4c19-9c17-96ca01ba3003 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:43.153359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b91664-fdeb-4c19-9c17-96ca01ba3003 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:43.153972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b91664-fdeb-4c19-9c17-96ca01ba3003 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:43.155404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b91664-fdeb-4c19-9c17-96ca01ba3003 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:43.163816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 465/912] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:03:42 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:43.729995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee456-e623-4a76-9101-66732f46b481 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:43.730539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee456-e623-4a76-9101-66732f46b481 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:43.730539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee456-e623-4a76-9101-66732f46b481 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:43.731791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 448/913] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:43.737402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffdd45-67e8-484f-83ba-938cd3258eb1 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:43.737690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffdd45-67e8-484f-83ba-938cd3258eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:43.738684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffdd45-67e8-484f-83ba-938cd3258eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:43.758443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-eeffdd45-67e8-484f-83ba-938cd3258eb1 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:43.759649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-eeffdd45-67e8-484f-83ba-938cd3258eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:43.785819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:43.837695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 466/914] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:44.109690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bee456-e623-4a76-9101-66732f46b481 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:44.130193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:44.214944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bee456-e623-4a76-9101-66732f46b481 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:44.249681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:44.250305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:44.250608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:44.250875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:44.251517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:44.252915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:44.262477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:44.263456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:44.264672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 449/915] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:44.270670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffdd45-67e8-484f-83ba-938cd3258eb1 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:44.377947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffdd45-67e8-484f-83ba-938cd3258eb1 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:44.414076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:44.415171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:44.415665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:44.416141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:44.416791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:44.417092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:44.420182 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:44.428631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 467/916] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:44.435915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:44.526836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:44.545665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a3fba-ed25-4e77-b4dc-cb5cbcdad786 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:44.580261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:44.580870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:44.581148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:44.582312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:44.582964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:44.583262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:44.584616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:44.589750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 450/917] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:44.595305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:44.622386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:44.700064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00594c-f87f-4c37-aa30-8351791a1abd None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:44.739731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb83128-d27d-4797-922a-f87ac95c754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:44.740825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb83128-d27d-4797-922a-f87ac95c754e None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:44.741280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb83128-d27d-4797-922a-f87ac95c754e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:44.741722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb83128-d27d-4797-922a-f87ac95c754e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:44.742620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb83128-d27d-4797-922a-f87ac95c754e None admin] RBAC: Adding query filter params (interface=public, service_id=8ef13c7499f1486cb737ef89ec6b4f92, region_id=RegionOne) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:44.743127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb83128-d27d-4797-922a-f87ac95c754e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:44.744711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb83128-d27d-4797-922a-f87ac95c754e None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:44.754204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 468/918] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Sep 9 23:03:44 2022] GET /identity/v3/endpoints?service_id=8ef13c7499f1486cb737ef89ec6b4f92&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:44.763475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:44.877589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02297354-2f84-44ba-aafb-5c6b551efe29 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:44.915663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79d518-fdaf-4daf-ba7d-fa749d033d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:44.916657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79d518-fdaf-4daf-ba7d-fa749d033d7c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:44.917138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79d518-fdaf-4daf-ba7d-fa749d033d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:44.917609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79d518-fdaf-4daf-ba7d-fa749d033d7c None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:44.918519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79d518-fdaf-4daf-ba7d-fa749d033d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:44.920132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79d518-fdaf-4daf-ba7d-fa749d033d7c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:44.927932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 451/919] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:03:44 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:45.246220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5f32-174b-4619-80f0-9f030764f554 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.246742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5f32-174b-4619-80f0-9f030764f554 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.247148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5f32-174b-4619-80f0-9f030764f554 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.249005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 469/920] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:45.254953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8c4a-54d1-4a56-845f-dc848cc4001b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.255376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8c4a-54d1-4a56-845f-dc848cc4001b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.255748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b8c4a-54d1-4a56-845f-dc848cc4001b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.275081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5a4b8c4a-54d1-4a56-845f-dc848cc4001b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:45.276259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5a4b8c4a-54d1-4a56-845f-dc848cc4001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.279309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:45.346370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 452/921] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:45.364457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e39d-f7da-4fb4-91aa-c4b9e112a753 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.364992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e39d-f7da-4fb4-91aa-c4b9e112a753 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.364992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969e39d-f7da-4fb4-91aa-c4b9e112a753 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.366584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 470/922] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:45.372420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc6ee-64bb-408a-8d75-d8665d534d3f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.372718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc6ee-64bb-408a-8d75-d8665d534d3f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.372969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebc6ee-64bb-408a-8d75-d8665d534d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.394907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f2ebc6ee-64bb-408a-8d75-d8665d534d3f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:45.396106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f2ebc6ee-64bb-408a-8d75-d8665d534d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.485506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 453/923] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:45.490367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d02358-36ee-41cb-add1-ef66bd6ecb16 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.490367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d02358-36ee-41cb-add1-ef66bd6ecb16 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.490367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d02358-36ee-41cb-add1-ef66bd6ecb16 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.490367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 471/924] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:45.494151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b4140-89c3-4393-b895-901c3d931fb6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.494782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b4140-89c3-4393-b895-901c3d931fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.495274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2b4140-89c3-4393-b895-901c3d931fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.515228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1c2b4140-89c3-4393-b895-901c3d931fb6 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:45.516699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1c2b4140-89c3-4393-b895-901c3d931fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:45.586799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:45.592215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 454/925] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:45.636138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d02358-36ee-41cb-add1-ef66bd6ecb16 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.742293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:45.757988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d02358-36ee-41cb-add1-ef66bd6ecb16 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.779439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b4140-89c3-4393-b895-901c3d931fb6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.793482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.794113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.794410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.794682 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.795261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:45.796616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:45.805691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:45.806742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:45.807999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 472/926] 10.222.0.20 () {56 vars in 1132 bytes} [Fri Sep 9 23:03:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:45.814517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:45.929168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2b4140-89c3-4393-b895-901c3d931fb6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.943248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d15477-6500-4f4d-af04-19364fb05d81 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:45.981556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1a8f3-c24a-4145-b6ac-4624857085b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.982572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1a8f3-c24a-4145-b6ac-4624857085b3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.983105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1a8f3-c24a-4145-b6ac-4624857085b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.983598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1a8f3-c24a-4145-b6ac-4624857085b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.996282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:45.997101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:45.997378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:45.997665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:45.998578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:45.999103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.000980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.004594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1a8f3-c24a-4145-b6ac-4624857085b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:46.009742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 473/927] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:46.010977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-01e1a8f3-c24a-4145-b6ac-4624857085b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:46.012110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:46.013152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:46.013995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 455/928] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Sep 9 23:03:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:46.022155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1a8f3-c24a-4145-b6ac-4624857085b3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.036207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.091937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.134406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e35f-5bea-46e5-936f-38033384cf68 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.143018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1a8f3-c24a-4145-b6ac-4624857085b3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.169366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.170325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.170806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.171284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.172153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.173774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.180388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.180988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.181264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.181534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.182157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:46.182484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.183830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:46.185212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:46.186167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 09 23:03:46.187195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:46.189121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 456/929] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:46.190393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.194648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 474/930] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:46.195346 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.286960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.299144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f73075-0247-483d-898c-f83dfbae8b34 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.313936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651fb0-90be-417a-b5aa-0e92ffd91e8c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.324050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.339171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.339777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.340048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.340314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.340958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:46.341259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.342662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.352888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 457/931] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Sep 9 23:03:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:46.353685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.368687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6171b0-2f04-4ca1-b5bd-324886ca9cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.369292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6171b0-2f04-4ca1-b5bd-324886ca9cff None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.369567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6171b0-2f04-4ca1-b5bd-324886ca9cff None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.369956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6171b0-2f04-4ca1-b5bd-324886ca9cff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.370648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6171b0-2f04-4ca1-b5bd-324886ca9cff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.371995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6171b0-2f04-4ca1-b5bd-324886ca9cff None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.433124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 475/932] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:46.443117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6171b0-2f04-4ca1-b5bd-324886ca9cff None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.483380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799f0a8-8db2-4935-9095-7f8710eb9024 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.530587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.531470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.531750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.532024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.532660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:46.532960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.534439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.545085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 458/933] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:46.546362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.588908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6171b0-2f04-4ca1-b5bd-324886ca9cff None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.609713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.632926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.632926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.633226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.633814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.636498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.649612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.656117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:46.666637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:46.667618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:46.668572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:46.668572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:46.668572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:46.668572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 476/934] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:46.674971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.688372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7f682-6a5a-4966-9455-8039c001954f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.727557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.728275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.728657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.728998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.729748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:46.730125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.731640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.737293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 459/935] 10.222.0.20 () {56 vars in 1114 bytes} [Fri Sep 9 23:03:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:46.746239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.797939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3cca8-24dd-4e13-99ab-4a30c5cc4029 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.846927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.848722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.849004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.849275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.849928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:46.850227 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.851632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.869416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50348807-1f6a-43b4-9a40-b85e9b8313e3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:46.874722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 477/936] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:46.875482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:46.888120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.921604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71593a23-05ee-409f-9248-75e60992eac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:46.922949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593a23-05ee-409f-9248-75e60992eac3 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:46.923404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593a23-05ee-409f-9248-75e60992eac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:46.924245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71593a23-05ee-409f-9248-75e60992eac3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:46.925475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71593a23-05ee-409f-9248-75e60992eac3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:46.927603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71593a23-05ee-409f-9248-75e60992eac3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:46.928874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:46.986767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992751b-3a4d-4d67-83f8-a84264b2eafa None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.023696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4b117-b653-4827-826b-5856a8c66b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.024319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b117-b653-4827-826b-5856a8c66b34 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.024596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b117-b653-4827-826b-5856a8c66b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.024860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4b117-b653-4827-826b-5856a8c66b34 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.032142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4b117-b653-4827-826b-5856a8c66b34 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.033591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4b117-b653-4827-826b-5856a8c66b34 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.046601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 460/937] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-20f4b117-b653-4827-826b-5856a8c66b34 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:47.053708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:47.054724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:47.055690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 478/938] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:03:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:47.063130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71593a23-05ee-409f-9248-75e60992eac3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.195389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71593a23-05ee-409f-9248-75e60992eac3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.237497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4223a-08ae-460e-8a6f-819bde89236e None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.237957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4223a-08ae-460e-8a6f-819bde89236e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.238161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4223a-08ae-460e-8a6f-819bde89236e None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.239708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 479/939] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:47.240919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.241826 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.242323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.242814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.244134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:47.244728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.245502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6f769-d4aa-434d-bae2-0dc67f80f331 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.245783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6f769-d4aa-434d-bae2-0dc67f80f331 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.246056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6f769-d4aa-434d-bae2-0dc67f80f331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.247190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.264862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f2c6f769-d4aa-434d-bae2-0dc67f80f331 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:47.266063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f2c6f769-d4aa-434d-bae2-0dc67f80f331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:47.272028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 461/940] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:47.277999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.303317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:47.359179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 480/941] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:47.401053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cada5-adc3-4686-af83-2c5e26f6e359 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.446080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.446733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.447007 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.447271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.458707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.460138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:47.476902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:47.477957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:47.478865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:47.478865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:47.478865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:47.478865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 462/942] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:47.483647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6f769-d4aa-434d-bae2-0dc67f80f331 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.519964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:47.587339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6f769-d4aa-434d-bae2-0dc67f80f331 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.621480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.622377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.622824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.623257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.624072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:47.624539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.626246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.638996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 481/943] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:47.646480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.666002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.725130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:47.759862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2eb931-163a-47a0-aed1-815185caa6cc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.777463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:47.786381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b99ab-22e7-468a-a62e-60f921465cbe None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.806155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.807265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.810133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.810133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.818636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.820838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.827183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:47.828004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:47.828343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:47.828680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:47.829324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:47.831049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:47.837897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:47.838907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:47.840021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:47.840021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:47.840021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:47.840021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:47.840021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:47.840021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:47.840021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:47.840021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:47.842087 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 463/944] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:47.842368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:47.843315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:47.844153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:47.847957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.849213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 482/945] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:47.850424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:47.964868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0674b5-a2c4-4fe9-8792-2becf33d1076 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:47.990804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaf576-fe9e-4151-aa2a-ff45f5de1388 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.004125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.004493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.004762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.005028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.005641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:48.006005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.007334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.017075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 483/946] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:48.018235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.028785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:48.041672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.042478 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.042756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.043018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.043705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:48.044002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.045422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.052088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:48.060068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 464/947] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:48.068179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.162944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a5234-6584-4736-9fec-46d9ab114801 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.184909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10712d3-6c61-45b5-a040-3df87d26299a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.211596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.212472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.213142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.213597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.215185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:48.215653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.218092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.222327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cea2d-8862-45eb-a207-997edc59aa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.223171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cea2d-8862-45eb-a207-997edc59aa92 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.223548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cea2d-8862-45eb-a207-997edc59aa92 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.223938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cea2d-8862-45eb-a207-997edc59aa92 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/2c8bfd34442b48a78600f568b534eaa1/roles/a65ba9a33bc44775a27c2cfdd3a54a42` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.225420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 484/948] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:48.230722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:48.231114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cea2d-8862-45eb-a207-997edc59aa92 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=2c8bfd34442b48a78600f568b534eaa1, role_id=a65ba9a33bc44775a27c2cfdd3a54a42)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.232563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cea2d-8862-45eb-a207-997edc59aa92 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.298463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 465/949] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:03:48 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/2c8bfd34442b48a78600f568b534eaa1/roles/a65ba9a33bc44775a27c2cfdd3a54a42 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:03:48.339203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:48.352429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3a8536-dda5-4a1b-8190-150affdd5d94 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:48.400234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eaf77c-97ad-4a1a-9355-c48596979d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:48.400853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf77c-97ad-4a1a-9355-c48596979d7f None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.401130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf77c-97ad-4a1a-9355-c48596979d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.401401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eaf77c-97ad-4a1a-9355-c48596979d7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.402146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaf77c-97ad-4a1a-9355-c48596979d7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:48.403718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eaf77c-97ad-4a1a-9355-c48596979d7f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:48.484891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 485/950] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:48.761474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c2e4c-f4af-442d-8a84-1046832274d7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.761772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c2e4c-f4af-442d-8a84-1046832274d7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.762053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c2e4c-f4af-442d-8a84-1046832274d7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.763715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 466/951] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:48.769950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d65e-2b86-4f96-9804-d3f25f4bd3a3 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:48.770233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d65e-2b86-4f96-9804-d3f25f4bd3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:48.770536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d65e-2b86-4f96-9804-d3f25f4bd3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:48.789629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-1069d65e-2b86-4f96-9804-d3f25f4bd3a3 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:48.790870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1069d65e-2b86-4f96-9804-d3f25f4bd3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:48.868835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 486/952] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:49.180107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c2e4c-f4af-442d-8a84-1046832274d7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:49.242737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:49.290368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c2e4c-f4af-442d-8a84-1046832274d7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:49.339012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bbce9f-6a6c-4c50-94b9-13d994e44093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:49.339683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbce9f-6a6c-4c50-94b9-13d994e44093 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:49.339963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbce9f-6a6c-4c50-94b9-13d994e44093 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:49.340235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bbce9f-6a6c-4c50-94b9-13d994e44093 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:49.340835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbce9f-6a6c-4c50-94b9-13d994e44093 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:49.344745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bbce9f-6a6c-4c50-94b9-13d994e44093 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-86bbce9f-6a6c-4c50-94b9-13d994e44093 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:49.357291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:49.358413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:49.359262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:49.362532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 467/953] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:49.365489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069d65e-2b86-4f96-9804-d3f25f4bd3a3 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:49.478186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:49.515570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069d65e-2b86-4f96-9804-d3f25f4bd3a3 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:49.532131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8447a0-eda6-4d84-94cd-90ff7ace0783 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:49.532447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8447a0-eda6-4d84-94cd-90ff7ace0783 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:49.532643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8447a0-eda6-4d84-94cd-90ff7ace0783 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:49.534148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 468/954] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:49.540999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1f0c-bffe-4d24-96b5-0db8daf11cf0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:49.540999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1f0c-bffe-4d24-96b5-0db8daf11cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:49.541277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1f0c-bffe-4d24-96b5-0db8daf11cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:49.567056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-21fb1f0c-bffe-4d24-96b5-0db8daf11cf0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:49.568483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-21fb1f0c-bffe-4d24-96b5-0db8daf11cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:49.623061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:49.623685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:49.623966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:49.624238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:49.624879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:49.625179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:49.627943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:49.637921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 487/955] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:49.647207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:49.687004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 469/956] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:49.770115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac5582-dacc-432e-9fa9-a29260cf4a6f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:49.806017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:49.806642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:49.806961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:49.807235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:49.807856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:49.808150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:49.809490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:49.810718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:49.818505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 488/957] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:49.960411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb1f0c-bffe-4d24-96b5-0db8daf11cf0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:50.012917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:50.068704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb1f0c-bffe-4d24-96b5-0db8daf11cf0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:50.104546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:50.105152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:50.105435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:50.105704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:50.106359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:50.107797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:50.110421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING keystone.server.flask.common [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 09 23:03:50.168229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.sql.core [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 23:03:50.168229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 23:03:50.168229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [parameters: {'id': '59e314374daa4d0087f72499694eaa51', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 23:03:50.168229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81769) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59e314374daa4d0087f72499694eaa51.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59e314374daa4d0087f72499694eaa51. Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application conn.query(q) Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 23:03:50.176764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result.read() Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 09 23:03:50.179430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application next(self.gen) Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application yield resource Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application next(self.gen) Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise self.value Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application session.commit() Sep 09 23:03:50.181125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.session.flush() Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application compat.raise_( Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 23:03:50.182017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 23:03:50.183014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application conn.query(q) Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result.read() Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 23:03:50.188165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [parameters: {'id': '59e314374daa4d0087f72499694eaa51', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:50.190247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:50.193540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 59e314374daa4d0087f72499694eaa51. Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:50.198052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 470/958] 10.222.0.20 () {60 vars in 1140 bytes} [Fri Sep 9 23:03:49 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:50.203900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:50.302707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3739d8-df5f-4e5e-8521-52a5588ed006 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:50.338627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff27741-86c5-45c4-b5a1-03d1f45cf44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:50.339574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff27741-86c5-45c4-b5a1-03d1f45cf44b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:50.340065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff27741-86c5-45c4-b5a1-03d1f45cf44b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:50.340529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff27741-86c5-45c4-b5a1-03d1f45cf44b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:50.347948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff27741-86c5-45c4-b5a1-03d1f45cf44b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:50.349679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff27741-86c5-45c4-b5a1-03d1f45cf44b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-4ff27741-86c5-45c4-b5a1-03d1f45cf44b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:50.361717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:50.362674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:50.364314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:50.364314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:50.364314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:50.364314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:50.364314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:50.364314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:50.364314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:50.364314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:50.366799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 489/959] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:50.372542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:50.490402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cd478-6921-4100-9faf-c28015940c5e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:50.492120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:50.529803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6811d8-0834-4968-90ee-cb0a88980901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:50.530468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6811d8-0834-4968-90ee-cb0a88980901 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:50.530802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6811d8-0834-4968-90ee-cb0a88980901 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:50.531083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6811d8-0834-4968-90ee-cb0a88980901 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:50.532016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6811d8-0834-4968-90ee-cb0a88980901 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:50.532324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6811d8-0834-4968-90ee-cb0a88980901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:50.534042 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6811d8-0834-4968-90ee-cb0a88980901 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:50.557305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 471/960] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:50.634038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893efc19-8410-4cc4-94fc-c15e6befc488 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:50.634570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893efc19-8410-4cc4-94fc-c15e6befc488 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:50.634570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893efc19-8410-4cc4-94fc-c15e6befc488 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:50.636194 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 490/961] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:50.642163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37940d92-33a0-4799-bdd7-1db7053aa4ff None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:50.642483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37940d92-33a0-4799-bdd7-1db7053aa4ff None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:50.642752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37940d92-33a0-4799-bdd7-1db7053aa4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:50.664105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-37940d92-33a0-4799-bdd7-1db7053aa4ff None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:50.665675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-37940d92-33a0-4799-bdd7-1db7053aa4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:50.749394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 472/962] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:50.826022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd54235-d964-4720-a9e8-f3e6812bc513 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:50.826304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd54235-d964-4720-a9e8-f3e6812bc513 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:50.826610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd54235-d964-4720-a9e8-f3e6812bc513 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:50.828200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 491/963] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:50.834043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da849-05bd-41f2-a539-5db94c781e91 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:50.834596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da849-05bd-41f2-a539-5db94c781e91 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:50.835063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7da849-05bd-41f2-a539-5db94c781e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:50.856992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1a7da849-05bd-41f2-a539-5db94c781e91 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:50.858201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1a7da849-05bd-41f2-a539-5db94c781e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:50.928778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 473/964] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:51.021477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd54235-d964-4720-a9e8-f3e6812bc513 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.083176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:51.127942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd54235-d964-4720-a9e8-f3e6812bc513 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.163317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ba284-5b9f-419f-91e2-e4c307b188e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.163942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ba284-5b9f-419f-91e2-e4c307b188e2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.164215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ba284-5b9f-419f-91e2-e4c307b188e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.164486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ba284-5b9f-419f-91e2-e4c307b188e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.175097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ba284-5b9f-419f-91e2-e4c307b188e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-ab5ba284-5b9f-419f-91e2-e4c307b188e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:03:51.180908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:51.181898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:51.183320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 492/965] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:51.191771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7da849-05bd-41f2-a539-5db94c781e91 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.212537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ba284-5b9f-419f-91e2-e4c307b188e2 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.217418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:51.305380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7da849-05bd-41f2-a539-5db94c781e91 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.352306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ba284-5b9f-419f-91e2-e4c307b188e2 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.353714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.354335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.354877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.355158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.355826 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:51.356161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:51.358510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:51.374221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 474/966] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:51.375630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.378043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:51.396420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.397209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.397453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.401494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.401494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:51.401494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:51.420284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:51.421364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:51.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:51.429486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 493/967] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:51.430636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.490197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:51.511399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea10f4-b655-41c4-bd86-79c4e62d2dc0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.555404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011eb5e4-7d0a-4a28-b248-5f2ea1545a8c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.559290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.560001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.560351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.560691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.574934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:51.576876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:51.592763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:51.593956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:51.595106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:51.595106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:51.595106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:51.601658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.601658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.601658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.601658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.602087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:51.602087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:51.602087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:51.602623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 475/968] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:51.603643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.607279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 494/969] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:51.613014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.667234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:51.719050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f33bf-a00d-40df-b49b-36349f7cebb7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.731662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d2a57-f9fa-4d88-b704-99e3e3994101 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.756407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.757292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.757773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.758226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.759138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:51.759640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:51.761459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:51.764205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:51.771049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.771672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.771956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.772213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.772842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:51.774803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:51.775279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:51.776973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 476/970] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:51.783376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.785595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 495/971] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:51.791382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.911857 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af5970-9b82-4194-ac25-ae0af7115f09 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.914503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85ae0c-9f8c-472d-95bd-304c18aaecd6 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:51.926811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:51.947232 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.948016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.948391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.948759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.958367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:51.958367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:51.958367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:51.958367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:51.959016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] RBAC: Adding query filter params (interface=admin, service_id=8ef13c7499f1486cb737ef89ec6b4f92, region_id=RegionOne) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:51.959016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:51.959016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:51.959260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:51.960752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:51.965534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 496/972] 10.222.0.20 () {56 vars in 1249 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/endpoints?service_id=8ef13c7499f1486cb737ef89ec6b4f92&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:51.973033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:51.973999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:51.975131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 09 23:03:51.976124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:51.980008 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 477/973] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:51.980862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:52.045383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:52.101620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c56e49-3a79-4cc0-a641-e8e9e5c0c696 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:52.101818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11e412-9ca0-4ad6-9a0f-0fec04ac6ca9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:52.143045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29db43-a43b-4ae8-b2c7-9e86f4a22843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:52.143652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29db43-a43b-4ae8-b2c7-9e86f4a22843 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:52.144082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29db43-a43b-4ae8-b2c7-9e86f4a22843 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:52.144356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29db43-a43b-4ae8-b2c7-9e86f4a22843 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:52.145185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29db43-a43b-4ae8-b2c7-9e86f4a22843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:52.146674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29db43-a43b-4ae8-b2c7-9e86f4a22843 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:52.150983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:52.151914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:52.152193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:52.152466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:52.153203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:52.153507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:52.153994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 497/974] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:52.155063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:52.155315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdea143-dc48-4956-8f7a-eeb842e20685 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:52.155604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdea143-dc48-4956-8f7a-eeb842e20685 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:52.155876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdea143-dc48-4956-8f7a-eeb842e20685 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:52.157475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 498/975] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:52.167042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67d236-ebf7-4fa9-93d2-2b430f9e5021 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:52.167042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67d236-ebf7-4fa9-93d2-2b430f9e5021 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:52.167339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67d236-ebf7-4fa9-93d2-2b430f9e5021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:52.180398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 478/976] 10.222.0.20 () {56 vars in 1149 bytes} [Fri Sep 9 23:03:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:52.190428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:52.190632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4b67d236-ebf7-4fa9-93d2-2b430f9e5021 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:52.190632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4b67d236-ebf7-4fa9-93d2-2b430f9e5021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:52.242830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:52.267071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 499/977] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:52.323215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82745277-030f-4619-8f49-af234bfe4b97 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:52.339585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:52.360046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bec46b-60de-47c6-a65a-d07c281c5dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:52.360656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec46b-60de-47c6-a65a-d07c281c5dde None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:52.360933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec46b-60de-47c6-a65a-d07c281c5dde None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:52.361198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec46b-60de-47c6-a65a-d07c281c5dde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:52.445009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bec46b-60de-47c6-a65a-d07c281c5dde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:52.446533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bec46b-60de-47c6-a65a-d07c281c5dde None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:52.546065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67d236-ebf7-4fa9-93d2-2b430f9e5021 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:52.553149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 479/978] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:03:52 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:03:52.558766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bec46b-60de-47c6-a65a-d07c281c5dde None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:52.574161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:52.666475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67d236-ebf7-4fa9-93d2-2b430f9e5021 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:52.671623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bec46b-60de-47c6-a65a-d07c281c5dde None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:52.708923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca880866-4d33-4993-8371-4aced093473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:52.709593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca880866-4d33-4993-8371-4aced093473f None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:52.709802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca880866-4d33-4993-8371-4aced093473f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:52.710073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca880866-4d33-4993-8371-4aced093473f None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:52.710759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca880866-4d33-4993-8371-4aced093473f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:52.712253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca880866-4d33-4993-8371-4aced093473f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:52.714881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:52.718542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:52.718542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:52.718542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:52.728903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:52.730629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:03:52.746811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:52.747914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:03:52.748829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:03:52.748829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:03:52.748829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:52.750487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 480/979] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:52.754181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:52.794672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 500/980] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:03:52 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:52.836319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:52.879606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952de49-0c39-460e-b7b3-770830b09def None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:52.931056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5227e-afef-4d56-b2ba-e9c50fbc2cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:52.931685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5227e-afef-4d56-b2ba-e9c50fbc2cbc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:52.931966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5227e-afef-4d56-b2ba-e9c50fbc2cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:52.932239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5227e-afef-4d56-b2ba-e9c50fbc2cbc None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:52.932900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5227e-afef-4d56-b2ba-e9c50fbc2cbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:52.933207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5227e-afef-4d56-b2ba-e9c50fbc2cbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:52.934681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5227e-afef-4d56-b2ba-e9c50fbc2cbc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:52.949596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 481/981] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:52.955326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca880866-4d33-4993-8371-4aced093473f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:52.987772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:53.011571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7af20-fb71-4305-b24e-4bc68a0cff13 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.011850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7af20-fb71-4305-b24e-4bc68a0cff13 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.012157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7af20-fb71-4305-b24e-4bc68a0cff13 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.015254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 482/982] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:53.020304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f159b7-d2ac-43ef-b391-66dc071e8a8d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.020592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f159b7-d2ac-43ef-b391-66dc071e8a8d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.020871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f159b7-d2ac-43ef-b391-66dc071e8a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.042665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a9f159b7-d2ac-43ef-b391-66dc071e8a8d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:53.043838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a9f159b7-d2ac-43ef-b391-66dc071e8a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:53.079999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca880866-4d33-4993-8371-4aced093473f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.117484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.118547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.119025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.119491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.127015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.134435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 483/983] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:53 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:53.134648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:03:53.144340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:53.145347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:53.146236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 501/984] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:53.152434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f159b7-d2ac-43ef-b391-66dc071e8a8d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.208837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:53.257448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f159b7-d2ac-43ef-b391-66dc071e8a8d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.291851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.292456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.292728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.292993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.293672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:53.294025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.295529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.308155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 484/985] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:03:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:53.314056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.420867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:53.422054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.435601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ec2f-1939-45ad-980a-c303df885ef3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.480959 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.481559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.481857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.482145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.482872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] RBAC: Adding query filter params (role.id=a65ba9a33bc44775a27c2cfdd3a54a42, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=2c8bfd34442b48a78600f568b534eaa1) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:53.483178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.484617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.492082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 502/986] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:03:53 2022] GET /identity/v3/role_assignments?user.id=2c8bfd34442b48a78600f568b534eaa1&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=a65ba9a33bc44775a27c2cfdd3a54a42 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:53.549878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1fcf0-b2ca-4e07-8d50-3508ce77e7a7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.564881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:53.588894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.589518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.589798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.590077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.590769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.592246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.642835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.sql.core [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 23:03:53.642835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 23:03:53.642835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [parameters: {'user_id': '2576351635c34e4ab0c5e6fa31029dac', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 23:03:53.642835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81769) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application conn.query(q) Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result.read() Sep 09 23:03:53.651511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 23:03:53.652900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application next(self.gen) Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application yield resource Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application next(self.gen) Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.session.rollback() Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.force_reraise() Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise self.value Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application session.commit() Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 09 23:03:53.653862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._prepare_impl() Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.session.flush() Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._flush(objects) Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application compat.raise_( Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application flush_context.execute() Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rec.execute(self) Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 09 23:03:53.654814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 09 23:03:53.655821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result = self._query(query) Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application conn.query(q) Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application result.read() Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 09 23:03:53.656662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [parameters: {'user_id': '2576351635c34e4ab0c5e6fa31029dac', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:53.657489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 09 23:03:53.661893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 09 23:03:53.664230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 09 23:03:53.664230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 09 23:03:53.664230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 09 23:03:53.664230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 09 23:03:53.664230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 09 23:03:53.664230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:53.664230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 485/987] 10.222.0.20 () {60 vars in 1131 bytes} [Fri Sep 9 23:03:53 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:53.669117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.782948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b6b23-67e8-4da8-9d71-6c11484b6f74 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:53.825042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552cfca-0eb5-4dcd-a01f-5d37dc1abe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:53.826062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552cfca-0eb5-4dcd-a01f-5d37dc1abe65 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:53.826575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552cfca-0eb5-4dcd-a01f-5d37dc1abe65 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:53.827053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552cfca-0eb5-4dcd-a01f-5d37dc1abe65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:53.835704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552cfca-0eb5-4dcd-a01f-5d37dc1abe65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:53.837482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552cfca-0eb5-4dcd-a01f-5d37dc1abe65 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-a552cfca-0eb5-4dcd-a01f-5d37dc1abe65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:03:53.850007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:53.850954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 09 23:03:53.851880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:53.854425 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 503/988] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:03:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:53.863090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:53.964982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e083440-b873-42d3-b95b-4774a2142444 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:54.001120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0939e-9fec-4924-9b6b-faf78018d804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:54.001974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0939e-9fec-4924-9b6b-faf78018d804 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:54.002416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0939e-9fec-4924-9b6b-faf78018d804 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:54.002825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0939e-9fec-4924-9b6b-faf78018d804 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:54.003823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0939e-9fec-4924-9b6b-faf78018d804 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:54.004342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0939e-9fec-4924-9b6b-faf78018d804 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:54.006115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0939e-9fec-4924-9b6b-faf78018d804 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:54.049347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 486/989] 10.222.0.20 () {56 vars in 1101 bytes} [Fri Sep 9 23:03:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:54.630110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239862-c438-4cff-8d20-b18e4d34ea3b None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:54.630500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239862-c438-4cff-8d20-b18e4d34ea3b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:54.630702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a239862-c438-4cff-8d20-b18e4d34ea3b None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:54.632046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 504/990] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:54.637220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2e60f-a093-41fe-82b0-716c9205d1eb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:54.637671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2e60f-a093-41fe-82b0-716c9205d1eb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:54.637749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2e60f-a093-41fe-82b0-716c9205d1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:54.655801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-12d2e60f-a093-41fe-82b0-716c9205d1eb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:54.657286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-12d2e60f-a093-41fe-82b0-716c9205d1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:54.682769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:54.724012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 487/991] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:54.992459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a239862-c438-4cff-8d20-b18e4d34ea3b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.009784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:55.100095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a239862-c438-4cff-8d20-b18e4d34ea3b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:55.115179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb3160-9db9-4f76-9aa7-de60e43fce8b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.115468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb3160-9db9-4f76-9aa7-de60e43fce8b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.115745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb3160-9db9-4f76-9aa7-de60e43fce8b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.117112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 488/992] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:55.122084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dce00-529d-490d-a9d0-fdc3cb651413 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.122390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dce00-529d-490d-a9d0-fdc3cb651413 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.122678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5dce00-529d-490d-a9d0-fdc3cb651413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.135344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:55.136389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.136671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.136941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.137685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:55.139064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:55.141118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2a5dce00-529d-490d-a9d0-fdc3cb651413 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:55.142322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2a5dce00-529d-490d-a9d0-fdc3cb651413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:55.148936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:55.149958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:55.154925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 505/993] 10.222.0.20 () {56 vars in 1126 bytes} [Fri Sep 9 23:03:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:55.158376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.213970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 489/994] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:55.248875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:55.272151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec506-1f4c-4f0e-a2df-5cbb594ea573 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:55.308065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:55.308666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.308943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.309210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.309827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:55.310123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:55.311508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:55.320938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 506/995] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:55.322736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5dce00-529d-490d-a9d0-fdc3cb651413 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.420346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:55.428311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5dce00-529d-490d-a9d0-fdc3cb651413 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:55.465437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:55.466039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.466311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.466606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.467224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:55.467518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:55.468870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:55.474379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 490/996] 10.222.0.20 () {56 vars in 1122 bytes} [Fri Sep 9 23:03:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:55.479935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.495332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.595845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6db7e3-6a63-4cf6-b3c3-850e01c662de None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:55.603786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6e572-09ae-4595-8331-5bd1d36a015d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:55.631369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a4c23-5bf3-44ef-b6f5-2cb09e97b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:55.631976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4c23-5bf3-44ef-b6f5-2cb09e97b470 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.632254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4c23-5bf3-44ef-b6f5-2cb09e97b470 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.632524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a4c23-5bf3-44ef-b6f5-2cb09e97b470 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.633195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a4c23-5bf3-44ef-b6f5-2cb09e97b470 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:55.634751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a4c23-5bf3-44ef-b6f5-2cb09e97b470 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:55.642259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:55.642913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.643195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.643444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.644559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:55.645898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:55.654751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:55.661895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:55.662785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 491/997] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:03:55.663582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.685069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:55.701741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:55.708291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 507/998] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:03:55.782299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd95012-fb66-4ee0-b804-d9dc0a587bc8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:55.822606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4ebae-ac99-47d4-b721-79e7259a35cf None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.822866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4ebae-ac99-47d4-b721-79e7259a35cf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.823134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4ebae-ac99-47d4-b721-79e7259a35cf None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.824505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 508/999] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:55.830152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c3389-4cbe-45bd-a222-a24caae2338d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.830486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c3389-4cbe-45bd-a222-a24caae2338d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.830774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c3389-4cbe-45bd-a222-a24caae2338d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.836835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:55.837456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:55.837734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:55.838004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:55.838721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:55.839020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:55.840388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:55.846029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 492/1000] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:55.851298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d88c3389-4cbe-45bd-a222-a24caae2338d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:55.852240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:55.853757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d88c3389-4cbe-45bd-a222-a24caae2338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:55.931153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 509/1001] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:55 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:55.948612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:55.966902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cad4e-73d7-4dee-a17a-8815eed72d18 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.000994 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.001590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.001861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.002128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.002783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:56.003075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.005154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.009892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 493/1002] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:56.015881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c3389-4cbe-45bd-a222-a24caae2338d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.118768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c3389-4cbe-45bd-a222-a24caae2338d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.147288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:56.153714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.154314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.154609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.154878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.155568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] RBAC: Adding query filter params (interface=public, service_id=b1ff77a2fad946419e3e547f9d8597ee, region_id=RegionOne) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:56.155867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.157396 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.163337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 510/1003] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Sep 9 23:03:56 2022] GET /identity/v3/endpoints?service_id=b1ff77a2fad946419e3e547f9d8597ee&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:03:56.168756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.202661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.277954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a010e-a5b3-42af-ae38-19853295f3f1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.312923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88170438-0105-417c-bcb5-a8a42910bda0 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.313463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3a055-0318-401c-922d-84304668a1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.314065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3a055-0318-401c-922d-84304668a1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.314352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3a055-0318-401c-922d-84304668a1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.314636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a3a055-0318-401c-922d-84304668a1e0 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.315296 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3a055-0318-401c-922d-84304668a1e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.317306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a3a055-0318-401c-922d-84304668a1e0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.323924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 494/1004] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:03:56 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:03:56.351574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.352232 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.352510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.352781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.353380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.354810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:56.365620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:56.366719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:03:56.371485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 511/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Sep 9 23:03:56 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:56.373232 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3a055-0318-401c-922d-84304668a1e0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.386939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:56.492254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a3a055-0318-401c-922d-84304668a1e0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.529878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4a9ce2-8d80-4881-a78b-38fc8b15dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.530594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9ce2-8d80-4881-a78b-38fc8b15dc0b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.530880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9ce2-8d80-4881-a78b-38fc8b15dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.531135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a9ce2-8d80-4881-a78b-38fc8b15dc0b None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.531767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9ce2-8d80-4881-a78b-38fc8b15dc0b None admin] RBAC: Adding query filter params (name=image) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:56.532066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9ce2-8d80-4881-a78b-38fc8b15dc0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.533414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4a9ce2-8d80-4881-a78b-38fc8b15dc0b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.539033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 495/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:56.544953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:56.606707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:56.647342 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c491f5-40a9-4da6-9874-92e36c4874c5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:56.684143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeddb4-990e-495a-9938-e8beba7eb8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:56.684768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeddb4-990e-495a-9938-e8beba7eb8af None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:56.685042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeddb4-990e-495a-9938-e8beba7eb8af None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:56.685309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeddb4-990e-495a-9938-e8beba7eb8af None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:56.685945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeddb4-990e-495a-9938-e8beba7eb8af None admin] RBAC: Adding query filter params (type=image) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:56.686243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeddb4-990e-495a-9938-e8beba7eb8af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:56.687733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaeddb4-990e-495a-9938-e8beba7eb8af None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:56.693060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 512/1007] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:57.955569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e2ea9-6bcc-41cf-b4b1-4bdc31891070 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.956092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e2ea9-6bcc-41cf-b4b1-4bdc31891070 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.956092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e2ea9-6bcc-41cf-b4b1-4bdc31891070 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.958039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 496/1008] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:57.963341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e580f5-c0b3-4a9a-94e2-32245c847edf None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:57.963791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e580f5-c0b3-4a9a-94e2-32245c847edf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:57.963874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e580f5-c0b3-4a9a-94e2-32245c847edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:57.982848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f9e580f5-c0b3-4a9a-94e2-32245c847edf None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:57.984058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f9e580f5-c0b3-4a9a-94e2-32245c847edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:58.053961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 513/1009] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:58.326782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e2ea9-6bcc-41cf-b4b1-4bdc31891070 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:58.399973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:58.444088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e2ea9-6bcc-41cf-b4b1-4bdc31891070 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:58.479477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b29e1c-206d-45ed-a6c3-62f3a269ffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:58.480102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29e1c-206d-45ed-a6c3-62f3a269ffae None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:58.480381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29e1c-206d-45ed-a6c3-62f3a269ffae None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:58.480654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b29e1c-206d-45ed-a6c3-62f3a269ffae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:58.481759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b29e1c-206d-45ed-a6c3-62f3a269ffae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:58.483209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b29e1c-206d-45ed-a6c3-62f3a269ffae None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:58.484953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 497/1010] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:03:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:58.491380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e580f5-c0b3-4a9a-94e2-32245c847edf None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:58.538492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87066d1e-217b-4cbf-9074-7f188c0b762c None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:58.538822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87066d1e-217b-4cbf-9074-7f188c0b762c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:58.539110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87066d1e-217b-4cbf-9074-7f188c0b762c None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:58.540490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 498/1011] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:03:58.545147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb181e-1b1e-480f-bfa0-6ccaf04161aa None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:58.545427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb181e-1b1e-480f-bfa0-6ccaf04161aa None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:58.545702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb181e-1b1e-480f-bfa0-6ccaf04161aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:58.546177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:58.564687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ebcb181e-1b1e-480f-bfa0-6ccaf04161aa None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:58.566497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ebcb181e-1b1e-480f-bfa0-6ccaf04161aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:58.598726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e580f5-c0b3-4a9a-94e2-32245c847edf None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:58.633937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767fe65-2277-4121-9e10-e328af1957d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:58.634573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767fe65-2277-4121-9e10-e328af1957d8 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:58.634853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767fe65-2277-4121-9e10-e328af1957d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:58.635161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767fe65-2277-4121-9e10-e328af1957d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:58.635774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767fe65-2277-4121-9e10-e328af1957d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:58.637253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 499/1012] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:03:58.637374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767fe65-2277-4121-9e10-e328af1957d8 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:58.724407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 514/1013] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:03:58 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:58.878232 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8dd21-1671-4e83-931f-4235dbf5b97d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:58.878976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8dd21-1671-4e83-931f-4235dbf5b97d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:58.879280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8dd21-1671-4e83-931f-4235dbf5b97d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:58.880643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 500/1014] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:03:58.886662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ee94-4f42-44f4-bb5d-29c401843360 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:58.887100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ee94-4f42-44f4-bb5d-29c401843360 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:58.887450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473ee94-4f42-44f4-bb5d-29c401843360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:58.907553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7473ee94-4f42-44f4-bb5d-29c401843360 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:03:58.908686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8dd21-1671-4e83-931f-4235dbf5b97d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:58.911028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7473ee94-4f42-44f4-bb5d-29c401843360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:03:58.990998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:58.995860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 515/1015] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:03:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:03:59.023020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8dd21-1671-4e83-931f-4235dbf5b97d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:59.058716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:59.059339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:59.059613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:59.059874 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:59.060443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:59.061782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:03:59.071423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:03:59.077657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:03:59.078556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 501/1016] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:03:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:03:59.079996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ee94-4f42-44f4-bb5d-29c401843360 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:59.115398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:59.181657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473ee94-4f42-44f4-bb5d-29c401843360 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:59.215793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:59.216393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:59.216671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:59.216929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:59.217534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:59.217828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:59.219255 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:59.230256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 516/1017] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:59.230472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:59.270706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:59.336804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c5ce0-8b91-46f1-8c1d-d1a909e85897 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:59.371620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:59.373864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:59.374490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:59.374766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:59.375032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:59.375618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59159b4f-d89a-4d5a-b8b0-70f09e82882b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:59.375719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:03:59.375931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:59.377279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:59.379221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:03:59.383771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 502/1018] 10.222.0.20 () {56 vars in 1112 bytes} [Fri Sep 9 23:03:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:03:59.389159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:03:59.412715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a836c9b-0616-44bb-995d-672cc7941851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:59.413328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a836c9b-0616-44bb-995d-672cc7941851 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:59.413602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a836c9b-0616-44bb-995d-672cc7941851 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:59.413872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a836c9b-0616-44bb-995d-672cc7941851 None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:59.414584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a836c9b-0616-44bb-995d-672cc7941851 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:59.415933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a836c9b-0616-44bb-995d-672cc7941851 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:59.496455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3e200-6a7b-48e5-8267-ce19ad39a9f9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:03:59.512938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 517/1019] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:03:59 2022] POST /identity/v3/services => generated 242 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:03:59.543320 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962ebb4-dfa1-407c-9354-2d6d285412f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:03:59.543947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962ebb4-dfa1-407c-9354-2d6d285412f5 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:03:59.544216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962ebb4-dfa1-407c-9354-2d6d285412f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:03:59.544482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962ebb4-dfa1-407c-9354-2d6d285412f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:03:59.545156 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962ebb4-dfa1-407c-9354-2d6d285412f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:03:59.546582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962ebb4-dfa1-407c-9354-2d6d285412f5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:03:59.630376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 503/1020] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:03:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:00.992797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34f6f-b5ab-450a-8fbf-ba3aa628af48 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:00.993331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34f6f-b5ab-450a-8fbf-ba3aa628af48 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:00.993331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34f6f-b5ab-450a-8fbf-ba3aa628af48 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:00.994746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 518/1021] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:00.999992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ac9cc-15f4-44b3-9632-01238792cf85 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:01.000439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ac9cc-15f4-44b3-9632-01238792cf85 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:01.000839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ac9cc-15f4-44b3-9632-01238792cf85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:01.022216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-470ac9cc-15f4-44b3-9632-01238792cf85 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:01.023429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-470ac9cc-15f4-44b3-9632-01238792cf85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:01.073748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:01.102047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 504/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:01.386834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad34f6f-b5ab-450a-8fbf-ba3aa628af48 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:01.504322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad34f6f-b5ab-450a-8fbf-ba3aa628af48 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:01.540332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad12674-e380-4b88-ab9a-adc29bae92f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:01.541299 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad12674-e380-4b88-ab9a-adc29bae92f7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:01.541729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad12674-e380-4b88-ab9a-adc29bae92f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:01.542152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad12674-e380-4b88-ab9a-adc29bae92f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:01.543445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad12674-e380-4b88-ab9a-adc29bae92f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:01.545025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad12674-e380-4b88-ab9a-adc29bae92f7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:01.547127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 519/1023] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:01.553170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ac9cc-15f4-44b3-9632-01238792cf85 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:01.676204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ac9cc-15f4-44b3-9632-01238792cf85 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:01.711081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19781dad-1bf5-4319-9254-21dcaf8adadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:01.711688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19781dad-1bf5-4319-9254-21dcaf8adadb None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:01.711963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19781dad-1bf5-4319-9254-21dcaf8adadb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:01.712231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19781dad-1bf5-4319-9254-21dcaf8adadb None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:01.712881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19781dad-1bf5-4319-9254-21dcaf8adadb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:01.714298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19781dad-1bf5-4319-9254-21dcaf8adadb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:01.734098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb962e-ee1f-4a7e-a5d0-1aeaa688cbc0 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:01.734519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb962e-ee1f-4a7e-a5d0-1aeaa688cbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:01.734799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb962e-ee1f-4a7e-a5d0-1aeaa688cbc0 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:01.736134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 520/1024] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:01.741699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a874a0-3c5f-4afd-86d9-70cc93f77476 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:01.741981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a874a0-3c5f-4afd-86d9-70cc93f77476 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:01.742252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a874a0-3c5f-4afd-86d9-70cc93f77476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:01.761239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-84a874a0-3c5f-4afd-86d9-70cc93f77476 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:01.762623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-84a874a0-3c5f-4afd-86d9-70cc93f77476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:01.782756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:01.828866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 521/1025] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:04:01.871378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 505/1026] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:04:01 2022] POST /identity/v3/users => generated 285 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:01.957822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d352fa8-86b2-4ece-be9e-a64b0c6a2b3b None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:01.958001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d352fa8-86b2-4ece-be9e-a64b0c6a2b3b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:01.958262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d352fa8-86b2-4ece-be9e-a64b0c6a2b3b None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:01.959819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 522/1027] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:01.965530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e9565-ec8a-4efe-93c7-7c2c27b899b3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:01.965820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e9565-ec8a-4efe-93c7-7c2c27b899b3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:01.966078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e9565-ec8a-4efe-93c7-7c2c27b899b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:01.986048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-347e9565-ec8a-4efe-93c7-7c2c27b899b3 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:01.987292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-347e9565-ec8a-4efe-93c7-7c2c27b899b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:02.016379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:02.067389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 506/1028] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:02.105888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d352fa8-86b2-4ece-be9e-a64b0c6a2b3b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.212103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d352fa8-86b2-4ece-be9e-a64b0c6a2b3b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:02.246683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:02.247300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:02.247571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:02.247837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] PATH_INFO: `/v3/services/image` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:02.248405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:02.249772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:02.258678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:02.259695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:02.260565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 523/1029] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:02.267330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e9565-ec8a-4efe-93c7-7c2c27b899b3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.339281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.367712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:02.374596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e9565-ec8a-4efe-93c7-7c2c27b899b3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:02.416741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:02.417404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:02.417680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:02.417944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:02.418584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:02.418881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:02.420425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:02.426686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 507/1030] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:02.432185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.467138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:02.467673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de627426-a8d8-456e-9ae6-fa724a082853 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:02.504213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9561a7-3b3c-4590-996e-28a6544c35b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:02.504907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9561a7-3b3c-4590-996e-28a6544c35b5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:02.505191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9561a7-3b3c-4590-996e-28a6544c35b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:02.505465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9561a7-3b3c-4590-996e-28a6544c35b5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:02.515952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9561a7-3b3c-4590-996e-28a6544c35b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-4b9561a7-3b3c-4590-996e-28a6544c35b5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:02.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:02.522114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:02.523140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 524/1031] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:02.527830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9561a7-3b3c-4590-996e-28a6544c35b5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.555090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b444786-43fd-4f8c-a502-c3d86424d6e4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:02.593739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:02.594819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:02.595224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:02.595498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:02.596123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] RBAC: Adding query filter params (type=image) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:02.596423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:02.598480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:02.606490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 508/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:02.610864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.617004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:02.637603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9561a7-3b3c-4590-996e-28a6544c35b5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:02.674482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:02.675091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:02.675820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:02.676095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:02.676760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:02.677061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:02.678426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:02.686655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 525/1033] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:02.692754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.714049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:02.724482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ddb214-1a1f-467e-adae-3f56ef215e8d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:02.761515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:02.762315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:02.762630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:02.762903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:02.763599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] RBAC: Adding query filter params (interface=public, service_id=55b1ee88ade4499894d6320c900b633c, region_id=RegionOne) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:02.763904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:02.765266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:02.771199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 509/1034] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/endpoints?service_id=55b1ee88ade4499894d6320c900b633c&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:02.776634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.800195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf742f-6442-474f-bee7-89372d3c6040 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:02.840941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:02.841575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:02.841855 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:02.842383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:02.853695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:02.855485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:02.857122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:04:02.870368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:02.871301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:04:02.872363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:04:02.872363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:02.872363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:02.872363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 526/1035] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:02.877287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:02.883958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e3db8-0d44-4d5d-b7bb-0ca591494af4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:02.919421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fcc25-4365-49dc-a0b5-ea8ed54e428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:02.920059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fcc25-4365-49dc-a0b5-ea8ed54e428e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:02.920330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fcc25-4365-49dc-a0b5-ea8ed54e428e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:02.920600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fcc25-4365-49dc-a0b5-ea8ed54e428e None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:02.921193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fcc25-4365-49dc-a0b5-ea8ed54e428e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:02.922606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9fcc25-4365-49dc-a0b5-ea8ed54e428e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:02.938382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 510/1036] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:04:03.000926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7dc2dd-f37c-4e25-9259-336d86ab797d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:03.038006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:03.038661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:03.038952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:03.039246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:03.039921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:03.040221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:03.041805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:03.056834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 527/1037] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:03.062852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fcc25-4365-49dc-a0b5-ea8ed54e428e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:03.085462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:03.171294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fcc25-4365-49dc-a0b5-ea8ed54e428e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:03.209615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:03.210242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:03.210544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:03.210809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:03.217914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:03.219355 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:04:03.230905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:03.231961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:03.232934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 511/1038] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:04:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:03.238391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:03.293815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:03.338935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add78f1d-6903-435c-b342-4a8f0ae99554 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:03.372395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:03.372991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:03.373262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:03.373530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:03.374241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:03.374566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:03.375963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:03.393074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 528/1039] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:04:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:03.398691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:03.485058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:03.507111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb5cb7-4f21-42d6-8c01-1794e7ce4281 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:03.541618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58518c-ad56-4e8d-b87f-2a425d13038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:03.542213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58518c-ad56-4e8d-b87f-2a425d13038f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:03.542517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58518c-ad56-4e8d-b87f-2a425d13038f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:03.542785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58518c-ad56-4e8d-b87f-2a425d13038f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:03.615697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58518c-ad56-4e8d-b87f-2a425d13038f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:03.617136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58518c-ad56-4e8d-b87f-2a425d13038f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:03.693255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:03.721864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 512/1040] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:03 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:03.726431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:03.825387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c89777-554d-4544-987a-9e882571af03 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:03.840953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:03.863269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:03.863898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:03.864175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:03.864446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:03.875028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:03.876397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:04:03.890764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:03.891718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:04:03.893265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:04:03.893265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:03.893265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:03.893265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 529/1041] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:03.897977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58518c-ad56-4e8d-b87f-2a425d13038f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:04.002829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58518c-ad56-4e8d-b87f-2a425d13038f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:04.042709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dd706-ee2d-4c58-ab56-f25200185535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:04.043330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd706-ee2d-4c58-ab56-f25200185535 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.043608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd706-ee2d-4c58-ab56-f25200185535 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.043877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd706-ee2d-4c58-ab56-f25200185535 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.044530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dd706-ee2d-4c58-ab56-f25200185535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:04.044837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dd706-ee2d-4c58-ab56-f25200185535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:04.046214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dd706-ee2d-4c58-ab56-f25200185535 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:04.063194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 513/1042] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:04.064839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:04.135177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca505801-a6ff-456c-804f-6689bdf68d2e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.135378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca505801-a6ff-456c-804f-6689bdf68d2e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.135638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca505801-a6ff-456c-804f-6689bdf68d2e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.136966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 514/1043] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:04:04.142435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a37b1-0fc4-4530-89ff-1ad6377e6f6a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.142652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a37b1-0fc4-4530-89ff-1ad6377e6f6a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.142914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a37b1-0fc4-4530-89ff-1ad6377e6f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.161318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f82a37b1-0fc4-4530-89ff-1ad6377e6f6a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:04.162526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f82a37b1-0fc4-4530-89ff-1ad6377e6f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:04.167569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c4a4c-79f7-410f-8afd-9ec21e46c3a5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:04.205705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:04.211715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:04.212315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.213361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.213361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.220195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:04.221635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:04:04.233834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:04.234765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:04.235663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 530/1044] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:04:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:04.240904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:04.250135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 515/1045] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:04:04.271108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:04.346523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3b745-e006-4d0e-9b96-b0163c03e13d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:04.381033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:04.381630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.381915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.382187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.382901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:04.383207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:04.384617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:04.396625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 531/1046] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:04:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:04.402380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a37b1-0fc4-4530-89ff-1ad6377e6f6a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:04.507025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a37b1-0fc4-4530-89ff-1ad6377e6f6a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:04.517117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:04.542131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:04.542727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.543006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.543280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.544000 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] RBAC: Adding query filter params (role.id=a65ba9a33bc44775a27c2cfdd3a54a42, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c3bec982c8474c349ca4901039844d26) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:04.544305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:04.545720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:04.552084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 516/1047] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:04 2022] GET /identity/v3/role_assignments?user.id=c3bec982c8474c349ca4901039844d26&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=a65ba9a33bc44775a27c2cfdd3a54a42 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:04.609722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:04.627713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5544891-520b-4347-a548-86f61ea0d61a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:04.665815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97673f-64d5-4f83-9478-37dbc1d4f2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:04.666382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97673f-64d5-4f83-9478-37dbc1d4f2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.666658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97673f-64d5-4f83-9478-37dbc1d4f2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.666923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97673f-64d5-4f83-9478-37dbc1d4f2a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.679049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97673f-64d5-4f83-9478-37dbc1d4f2a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-fd97673f-64d5-4f83-9478-37dbc1d4f2a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:04.683946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:04.684924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:04.685755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:04.690221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 532/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:04:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:04.691584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:04.727080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:04.797388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a63f1-15f0-4a92-acd1-13d3848fe761 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:04.830975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:04.832344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.832614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.833043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.833694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:04.833984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:04.835361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:04.843868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 517/1049] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:04:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:04.850459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97673f-64d5-4f83-9478-37dbc1d4f2a5 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:04.949591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97673f-64d5-4f83-9478-37dbc1d4f2a5 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:04.983573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c277100-2ce9-4d68-bab7-81d37d49922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:04.984193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c277100-2ce9-4d68-bab7-81d37d49922a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:04.984466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c277100-2ce9-4d68-bab7-81d37d49922a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:04.984733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c277100-2ce9-4d68-bab7-81d37d49922a None admin] PATH_INFO: `/v3/users/32e9f6f2673e4efaa26f98913e9ba77f` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:04.998236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c277100-2ce9-4d68-bab7-81d37d49922a None admin] RBAC: Authorizing `identity:get_user(user_id=32e9f6f2673e4efaa26f98913e9ba77f)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:04.999692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c277100-2ce9-4d68-bab7-81d37d49922a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:05.006844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 533/1050] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:04 2022] GET /identity/v3/users/32e9f6f2673e4efaa26f98913e9ba77f => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:05.007834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:05.105996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:05.113787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a21e2-2fe6-4852-a799-d93784c04135 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:05.142283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd661b3-c88f-4489-af67-eadfffae25f7 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:05.142585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd661b3-c88f-4489-af67-eadfffae25f7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:05.142850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd661b3-c88f-4489-af67-eadfffae25f7 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:05.145226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 534/1051] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:04:05.151609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40311f-7bca-46f6-acf3-ea08eff0cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:05.151609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40311f-7bca-46f6-acf3-ea08eff0cb45 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:05.151845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6379dd4-e064-4b51-8146-4673ea7ac589 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:05.151951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40311f-7bca-46f6-acf3-ea08eff0cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:05.152051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6379dd4-e064-4b51-8146-4673ea7ac589 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:05.152135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40311f-7bca-46f6-acf3-ea08eff0cb45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:05.152224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6379dd4-e064-4b51-8146-4673ea7ac589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:05.158651 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:05.170996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d6379dd4-e064-4b51-8146-4673ea7ac589 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:05.172668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d6379dd4-e064-4b51-8146-4673ea7ac589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:05.222620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb40311f-7bca-46f6-acf3-ea08eff0cb45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:05.223986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb40311f-7bca-46f6-acf3-ea08eff0cb45 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:05.247801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 535/1052] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:04:05.318612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:05.330215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 518/1053] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 23:04:05.335919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6379dd4-e064-4b51-8146-4673ea7ac589 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:05.436038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6379dd4-e064-4b51-8146-4673ea7ac589 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:05.467179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:05.472226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80986a6a-1368-445f-909b-4b6feacb8bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:05.472978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80986a6a-1368-445f-909b-4b6feacb8bbe None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:05.473203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80986a6a-1368-445f-909b-4b6feacb8bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:05.473471 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80986a6a-1368-445f-909b-4b6feacb8bbe None admin] PATH_INFO: `/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:05.474629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80986a6a-1368-445f-909b-4b6feacb8bbe None admin] RBAC: Authorizing `identity:get_project(project_id=ced8a31d51514742a0fc8f38b1d3f96b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:05.476044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80986a6a-1368-445f-909b-4b6feacb8bbe None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:05.477748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 536/1054] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:05 2022] GET /identity/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:05.483642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40311f-7bca-46f6-acf3-ea08eff0cb45 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:05.518905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80986a6a-1368-445f-909b-4b6feacb8bbe None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:05.586820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40311f-7bca-46f6-acf3-ea08eff0cb45 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:05.619775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:05.620378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:05.620650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:05.620915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:05.621623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=ced8a31d51514742a0fc8f38b1d3f96b, user.id=32e9f6f2673e4efaa26f98913e9ba77f) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:05.621922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:05.623354 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:05.629476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 519/1055] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:05 2022] GET /identity/v3/role_assignments?user.id=32e9f6f2673e4efaa26f98913e9ba77f&scope.project.id=ced8a31d51514742a0fc8f38b1d3f96b&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:05.643368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80986a6a-1368-445f-909b-4b6feacb8bbe None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:05.683399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:05.684042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:05.684313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:05.684577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] PATH_INFO: `/v3/services/image` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:05.685157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:05.686570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:05.697035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:05.698093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:05.703412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 537/1056] 10.222.0.20 () {56 vars in 1105 bytes} [Fri Sep 9 23:04:05 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:05.704914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:05.770861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:05.816687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20824ab0-c1a6-4a31-b3cc-bcc3b78f6047 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:05.852416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:05.853021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:05.853299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:05.853572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:05.854198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] RBAC: Adding query filter params (name=image) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:05.854527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:05.855889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:05.865572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 520/1057] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:05.867107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:05.931408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:05.969892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d8b2-c8ba-4840-976c-77351abf8e9d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:06.004548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7be65-f4dc-47d1-bc6d-3101d31ccf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:06.005142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7be65-f4dc-47d1-bc6d-3101d31ccf8f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:06.005417 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7be65-f4dc-47d1-bc6d-3101d31ccf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:06.005679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7be65-f4dc-47d1-bc6d-3101d31ccf8f None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:06.006287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7be65-f4dc-47d1-bc6d-3101d31ccf8f None admin] RBAC: Adding query filter params (type=image) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:06.006639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7be65-f4dc-47d1-bc6d-3101d31ccf8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:06.008055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7be65-f4dc-47d1-bc6d-3101d31ccf8f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:06.017557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 538/1058] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:06.018783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:06.123058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98aa05-e0bb-46ce-8424-60d39d519412 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:06.146268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:06.158177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a3f66-95b5-4df1-a8bc-93b7fc5da82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:06.158824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a3f66-95b5-4df1-a8bc-93b7fc5da82b None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:06.159103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a3f66-95b5-4df1-a8bc-93b7fc5da82b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:06.159372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a3f66-95b5-4df1-a8bc-93b7fc5da82b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:06.160360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a3f66-95b5-4df1-a8bc-93b7fc5da82b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:06.161737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a3f66-95b5-4df1-a8bc-93b7fc5da82b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:06.256753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 521/1059] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:04:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:04:06.722786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8190540-3687-4192-ab2c-8f4fe19ee8ef None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:06.722786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8190540-3687-4192-ab2c-8f4fe19ee8ef None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:06.723343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8190540-3687-4192-ab2c-8f4fe19ee8ef None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:06.724374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 539/1060] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:06.729588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15742c8-24a3-4364-a974-e60b530e2d35 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:06.729871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15742c8-24a3-4364-a974-e60b530e2d35 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:06.730142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15742c8-24a3-4364-a974-e60b530e2d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:06.750216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d15742c8-24a3-4364-a974-e60b530e2d35 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:06.751440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d15742c8-24a3-4364-a974-e60b530e2d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:06.828430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 522/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:07.101507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8190540-3687-4192-ab2c-8f4fe19ee8ef None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:07.210956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8190540-3687-4192-ab2c-8f4fe19ee8ef None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:07.237788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:07.246603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eace9b-f818-47b9-a20a-6afb4dc04e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:07.247236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eace9b-f818-47b9-a20a-6afb4dc04e88 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:07.247512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eace9b-f818-47b9-a20a-6afb4dc04e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:07.247779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eace9b-f818-47b9-a20a-6afb4dc04e88 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:07.259045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eace9b-f818-47b9-a20a-6afb4dc04e88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-66eace9b-f818-47b9-a20a-6afb4dc04e88 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:07.263814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:07.264728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:04:07.265554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:07.270638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 540/1062] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Sep 9 23:04:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:07.272246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15742c8-24a3-4364-a974-e60b530e2d35 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:07.372855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15742c8-24a3-4364-a974-e60b530e2d35 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:07.376288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:07.409135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a1045-0186-446c-992b-da920b1e46de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:07.410370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a1045-0186-446c-992b-da920b1e46de None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:07.410900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a1045-0186-446c-992b-da920b1e46de None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:07.411371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a1045-0186-446c-992b-da920b1e46de None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:07.412234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a1045-0186-446c-992b-da920b1e46de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:07.412732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a1045-0186-446c-992b-da920b1e46de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:07.414309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a1045-0186-446c-992b-da920b1e46de None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:07.430491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 523/1063] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Sep 9 23:04:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:07.431613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eace9b-f818-47b9-a20a-6afb4dc04e88 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:07.536751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eace9b-f818-47b9-a20a-6afb4dc04e88 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:07.571553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:07.572178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:07.572452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:07.572719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:07.583107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:07.584528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:04:07.600975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:07.601954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:04:07.603387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:04:07.603387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:07.603387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:07.609292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 541/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:07.610851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a1045-0186-446c-992b-da920b1e46de None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:07.779424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a1045-0186-446c-992b-da920b1e46de None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:07.836432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d540a-9c06-432c-bc7b-c640cdda37fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:07.837483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d540a-9c06-432c-bc7b-c640cdda37fb None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:07.837956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d540a-9c06-432c-bc7b-c640cdda37fb None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:07.838507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d540a-9c06-432c-bc7b-c640cdda37fb None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:07.839626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d540a-9c06-432c-bc7b-c640cdda37fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:07.840161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d540a-9c06-432c-bc7b-c640cdda37fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:07.842670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d540a-9c06-432c-bc7b-c640cdda37fb None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:07.867458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 524/1065] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:07.868760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:07.907553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e038-0bb2-4b9d-899d-05a7e039765d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:07.907860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e038-0bb2-4b9d-899d-05a7e039765d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:07.908092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8e038-0bb2-4b9d-899d-05a7e039765d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:07.909631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 525/1066] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:04:07.914290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ffd1-000a-4c5a-8069-3955677e755d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:07.914602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ffd1-000a-4c5a-8069-3955677e755d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:07.914736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:07.914902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ffd1-000a-4c5a-8069-3955677e755d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:07.932571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c339ffd1-000a-4c5a-8069-3955677e755d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:07.933736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c339ffd1-000a-4c5a-8069-3955677e755d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:07.960433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:07.979098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604e910-5c61-4f02-b4b0-8118f8ef0dc7 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.011081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 526/1067] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:04:08.013852 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.014523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.014802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.015071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.023237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:08.024667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:04:08.036759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:08.037688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:08.038696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 542/1068] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:04:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:08.043620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339ffd1-000a-4c5a-8069-3955677e755d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.145304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339ffd1-000a-4c5a-8069-3955677e755d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.179219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.179823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.180100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.180368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.181112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:08.181412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:08.182874 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:08.204836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 527/1069] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:08.206364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.287282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.303580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:08.312962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:08.316633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c16592-e9ec-46a1-b7cd-030fd698bfbe None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.355970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.356617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.356889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.357153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.371440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:08.372957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:04:08.391037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:08.392046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:04:08.392975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:04:08.392975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:08.392975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:08.392975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 543/1070] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:08.398483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.401248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af3fc9-15ab-479d-94aa-b711f3cbdedf None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.439886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc7d12-d383-4b3f-afdf-c57f2dd408b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.440553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc7d12-d383-4b3f-afdf-c57f2dd408b8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.440938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc7d12-d383-4b3f-afdf-c57f2dd408b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.441228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc7d12-d383-4b3f-afdf-c57f2dd408b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.452131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc7d12-d383-4b3f-afdf-c57f2dd408b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-b0fc7d12-d383-4b3f-afdf-c57f2dd408b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:08.458034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:08.459128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:08.464891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 528/1071] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:08.466885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc7d12-d383-4b3f-afdf-c57f2dd408b8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.514911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d8bbe-c2c6-49ab-a7b7-77941385dc0f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.541567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:08.547436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004999a-11f7-436e-a950-cf9604e4347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.548034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004999a-11f7-436e-a950-cf9604e4347a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.548308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004999a-11f7-436e-a950-cf9604e4347a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.548584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004999a-11f7-436e-a950-cf9604e4347a None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.549232 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004999a-11f7-436e-a950-cf9604e4347a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:08.549505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004999a-11f7-436e-a950-cf9604e4347a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:08.551502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004999a-11f7-436e-a950-cf9604e4347a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:08.563030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 544/1072] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:08.564198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bc8a1-d8b2-4ae6-b8ed-ed75ce9ce1d0 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.564484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bc8a1-d8b2-4ae6-b8ed-ed75ce9ce1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.564753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bc8a1-d8b2-4ae6-b8ed-ed75ce9ce1d0 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.566400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 545/1073] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:08.568812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bc8a1-d8b2-4ae6-b8ed-ed75ce9ce1d0 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.571028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc7d12-d383-4b3f-afdf-c57f2dd408b8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.584889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:08.606313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a626290-8776-4b1b-a6a4-3e640c8ccc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.607032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a626290-8776-4b1b-a6a4-3e640c8ccc2b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.607313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a626290-8776-4b1b-a6a4-3e640c8ccc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.607579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a626290-8776-4b1b-a6a4-3e640c8ccc2b None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.608229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a626290-8776-4b1b-a6a4-3e640c8ccc2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:08.608526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a626290-8776-4b1b-a6a4-3e640c8ccc2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:08.609874 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a626290-8776-4b1b-a6a4-3e640c8ccc2b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:08.617845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 529/1074] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:08.619036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d28b09-4402-49b7-a87e-8df2013890cc None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.619319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d28b09-4402-49b7-a87e-8df2013890cc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.619593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d28b09-4402-49b7-a87e-8df2013890cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.637507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-04d28b09-4402-49b7-a87e-8df2013890cc None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:08.638730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-04d28b09-4402-49b7-a87e-8df2013890cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:08.673415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053bc8a1-d8b2-4ae6-b8ed-ed75ce9ce1d0 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.705073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 530/1075] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:08.705896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d28b09-4402-49b7-a87e-8df2013890cc None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.714182 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.716814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.722555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.722555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.724795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:08.726223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:04:08.738321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:08.739322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:08.740208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 546/1076] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:08.745411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.809821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:08.848280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d28b09-4402-49b7-a87e-8df2013890cc None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.849783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:08.880352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d53db-c68d-4f6f-86a1-3c1751c0ddf3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:08.882679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3082d8-f3f5-40ef-b59e-def5603c9217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.883307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3082d8-f3f5-40ef-b59e-def5603c9217 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.883586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3082d8-f3f5-40ef-b59e-def5603c9217 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.883853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3082d8-f3f5-40ef-b59e-def5603c9217 None admin] PATH_INFO: `/v3/users/32e9f6f2673e4efaa26f98913e9ba77f` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.884983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3082d8-f3f5-40ef-b59e-def5603c9217 None admin] RBAC: Authorizing `identity:get_user(user_id=32e9f6f2673e4efaa26f98913e9ba77f)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:08.886424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3082d8-f3f5-40ef-b59e-def5603c9217 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:08.888015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 531/1077] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/users/32e9f6f2673e4efaa26f98913e9ba77f => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:08.893503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3082d8-f3f5-40ef-b59e-def5603c9217 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.922616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:08.923425 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:08.923714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:08.924300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:08.925421 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:08.925791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:08.927239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:08.945409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 547/1078] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:08.947122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:08.992967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3082d8-f3f5-40ef-b59e-def5603c9217 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:09.031756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91b742-eadd-4daa-b442-a71098b6a931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:09.032379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b742-eadd-4daa-b442-a71098b6a931 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:09.032658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b742-eadd-4daa-b442-a71098b6a931 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:09.032920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b742-eadd-4daa-b442-a71098b6a931 None admin] PATH_INFO: `/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:09.034024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91b742-eadd-4daa-b442-a71098b6a931 None admin] RBAC: Authorizing `identity:get_project(project_id=ced8a31d51514742a0fc8f38b1d3f96b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:09.036315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91b742-eadd-4daa-b442-a71098b6a931 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:09.037948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 532/1079] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:08 2022] GET /identity/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:09.038722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91b742-eadd-4daa-b442-a71098b6a931 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:09.041309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:09.056095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc83f19-9590-4546-a3e8-bcc13f19d1cd None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:09.085811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:09.091365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f033947-640b-42d6-a2ca-ca030d08529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:09.092047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f033947-640b-42d6-a2ca-ca030d08529e None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:09.092323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f033947-640b-42d6-a2ca-ca030d08529e None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:09.092587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f033947-640b-42d6-a2ca-ca030d08529e None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c3bec982c8474c349ca4901039844d26/roles/a65ba9a33bc44775a27c2cfdd3a54a42` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:09.094721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f033947-640b-42d6-a2ca-ca030d08529e None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=c3bec982c8474c349ca4901039844d26, role_id=a65ba9a33bc44775a27c2cfdd3a54a42)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:09.097767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f033947-640b-42d6-a2ca-ca030d08529e None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:09.137693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91b742-eadd-4daa-b442-a71098b6a931 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:09.171150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a351d9a-63ec-412e-b564-e2cd2e628f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:09.171754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a351d9a-63ec-412e-b564-e2cd2e628f59 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:09.172031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a351d9a-63ec-412e-b564-e2cd2e628f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:09.172304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a351d9a-63ec-412e-b564-e2cd2e628f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:09.173799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 548/1080] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:04:08 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/c3bec982c8474c349ca4901039844d26/roles/a65ba9a33bc44775a27c2cfdd3a54a42 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:09.174639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f033947-640b-42d6-a2ca-ca030d08529e None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:09.260256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a351d9a-63ec-412e-b564-e2cd2e628f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:09.262053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a351d9a-63ec-412e-b564-e2cd2e628f59 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:09.264190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:09.295432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f033947-640b-42d6-a2ca-ca030d08529e None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:09.333316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c36c1-bfd1-4e4d-92c3-ca04c5a74da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:09.333997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c36c1-bfd1-4e4d-92c3-ca04c5a74da4 None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:09.334282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c36c1-bfd1-4e4d-92c3-ca04c5a74da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:09.334757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c36c1-bfd1-4e4d-92c3-ca04c5a74da4 None admin] PATH_INFO: `/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b/users/32e9f6f2673e4efaa26f98913e9ba77f/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:09.336835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c36c1-bfd1-4e4d-92c3-ca04c5a74da4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ced8a31d51514742a0fc8f38b1d3f96b, user_id=32e9f6f2673e4efaa26f98913e9ba77f, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:09.338214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c36c1-bfd1-4e4d-92c3-ca04c5a74da4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:09.380687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 533/1081] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:09.387664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a351d9a-63ec-412e-b564-e2cd2e628f59 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:09.424253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 549/1082] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:04:09 2022] PUT /identity/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b/users/32e9f6f2673e4efaa26f98913e9ba77f/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:09.503086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a351d9a-63ec-412e-b564-e2cd2e628f59 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:09.505377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:09.541471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca9f2c-d806-4bb9-ab0e-06dedf8b6842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:09.542108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca9f2c-d806-4bb9-ab0e-06dedf8b6842 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:09.542400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca9f2c-d806-4bb9-ab0e-06dedf8b6842 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:09.542666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca9f2c-d806-4bb9-ab0e-06dedf8b6842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:09.554032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca9f2c-d806-4bb9-ab0e-06dedf8b6842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:09.555526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca9f2c-d806-4bb9-ab0e-06dedf8b6842 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-efca9f2c-d806-4bb9-ab0e-06dedf8b6842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:04:09.572214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:09.573464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:04:09.574649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:04:09.574649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:09.574649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:09.574649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 534/1083] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:09.580262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c36c1-bfd1-4e4d-92c3-ca04c5a74da4 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:09.639964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:09.689153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c36c1-bfd1-4e4d-92c3-ca04c5a74da4 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:09.725321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d621a-24b3-46db-9306-44a8e7bbaf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:09.726028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d621a-24b3-46db-9306-44a8e7bbaf94 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:09.726403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d621a-24b3-46db-9306-44a8e7bbaf94 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:09.726791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d621a-24b3-46db-9306-44a8e7bbaf94 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:09.727508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d621a-24b3-46db-9306-44a8e7bbaf94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:09.727881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d621a-24b3-46db-9306-44a8e7bbaf94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:09.729310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d621a-24b3-46db-9306-44a8e7bbaf94 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:09.741882 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 550/1084] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:11.335215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5a67c-150e-43f7-9c65-ccde379d7fe1 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:11.335862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5a67c-150e-43f7-9c65-ccde379d7fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:11.335862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5a67c-150e-43f7-9c65-ccde379d7fe1 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:11.337100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 535/1085] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:11.342462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24980097-5e36-44cd-b840-d316436dce4e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:11.342886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24980097-5e36-44cd-b840-d316436dce4e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:11.343011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24980097-5e36-44cd-b840-d316436dce4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:11.361229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-24980097-5e36-44cd-b840-d316436dce4e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:11.362419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-24980097-5e36-44cd-b840-d316436dce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:11.430506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 551/1086] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:11.606573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959ae6-eec5-4c81-b17e-c59585fb80c2 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:11.606766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959ae6-eec5-4c81-b17e-c59585fb80c2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:11.607028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e959ae6-eec5-4c81-b17e-c59585fb80c2 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:11.608356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 536/1087] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:11.613757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ebd3-c3d8-4e00-a3a7-426289a3e11e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:11.614043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ebd3-c3d8-4e00-a3a7-426289a3e11e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:11.614313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7ebd3-c3d8-4e00-a3a7-426289a3e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:11.636564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e0f7ebd3-c3d8-4e00-a3a7-426289a3e11e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:11.637785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e0f7ebd3-c3d8-4e00-a3a7-426289a3e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:11.672753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:11.703686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 552/1088] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:11.715526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e959ae6-eec5-4c81-b17e-c59585fb80c2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:11.819414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e959ae6-eec5-4c81-b17e-c59585fb80c2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:11.853733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a22550-a000-4a55-bb77-9b080a96a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:11.854396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a22550-a000-4a55-bb77-9b080a96a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:11.854675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a22550-a000-4a55-bb77-9b080a96a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:11.854944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a22550-a000-4a55-bb77-9b080a96a0f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:11.865765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a22550-a000-4a55-bb77-9b080a96a0f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-30a22550-a000-4a55-bb77-9b080a96a0f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:11.870509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:11.871672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 09 23:04:11.872584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:11.873593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 537/1089] 10.222.0.20 () {56 vars in 1120 bytes} [Fri Sep 9 23:04:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:11.877513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ebd3-c3d8-4e00-a3a7-426289a3e11e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:11.970579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a22550-a000-4a55-bb77-9b080a96a0f5 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:11.979112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7ebd3-c3d8-4e00-a3a7-426289a3e11e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:11.986754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.013157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.013768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.014056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.014668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.015337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:12.015644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.017189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.026018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 553/1090] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Sep 9 23:04:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:12.031805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.065942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.085901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a22550-a000-4a55-bb77-9b080a96a0f5 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.120257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7241fb-408d-48a0-ba52-aa2231a9a9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.120802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7241fb-408d-48a0-ba52-aa2231a9a9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.121083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7241fb-408d-48a0-ba52-aa2231a9a9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.121354 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7241fb-408d-48a0-ba52-aa2231a9a9b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.131489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7241fb-408d-48a0-ba52-aa2231a9a9b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:12.136322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61af1d9-40e9-4999-ab3c-a975990db3b9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-9c7241fb-408d-48a0-ba52-aa2231a9a9b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:12.136457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:12.137502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:12.138369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 538/1091] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:04:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:12.143302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7241fb-408d-48a0-ba52-aa2231a9a9b9 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.172594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.173210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.173491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.173757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.183649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.185068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:04:12.199763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:12.200745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:04:12.201645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:04:12.201645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:12.201645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:12.201645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 554/1092] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:12.207350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.218424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.242772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7241fb-408d-48a0-ba52-aa2231a9a9b9 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.280070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.280666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.280934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.281197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.281849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:12.282142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.283526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.292312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 539/1093] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:12.297937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.299180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.314810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2323d3-d3a1-49bb-bfab-dde1c4028f4c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.351622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.352289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.352479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.352751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.353379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:12.353712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.355090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.366560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 555/1094] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:12.371804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.398200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3cc63-8074-4128-996c-42e744895f13 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.431928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1883-51cb-4f32-98f0-68e8af80e742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.432563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1883-51cb-4f32-98f0-68e8af80e742 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.432840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1883-51cb-4f32-98f0-68e8af80e742 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.433111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da1883-51cb-4f32-98f0-68e8af80e742 None admin] PATH_INFO: `/v3/users/32e9f6f2673e4efaa26f98913e9ba77f` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.434245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da1883-51cb-4f32-98f0-68e8af80e742 None admin] RBAC: Authorizing `identity:get_user(user_id=32e9f6f2673e4efaa26f98913e9ba77f)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.435718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da1883-51cb-4f32-98f0-68e8af80e742 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.437302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 540/1095] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/users/32e9f6f2673e4efaa26f98913e9ba77f => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:12.443490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1883-51cb-4f32-98f0-68e8af80e742 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.446923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.473940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5603c-2598-44a9-b5bf-48cfb883b4b0 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.508742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.509365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.509642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.509911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.517160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.518619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:04:12.530567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:12.531650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:12.532585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 556/1096] 10.222.0.20 () {56 vars in 1093 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:12.537483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.541444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da1883-51cb-4f32-98f0-68e8af80e742 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.546096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.576521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f0729-feff-49ca-b458-6baa022f9302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.577122 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0729-feff-49ca-b458-6baa022f9302 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.577395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0729-feff-49ca-b458-6baa022f9302 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.577661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0729-feff-49ca-b458-6baa022f9302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.637386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6e46d-8716-446e-ab76-b3e577dace6d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.643343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f0729-feff-49ca-b458-6baa022f9302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.645335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f0729-feff-49ca-b458-6baa022f9302 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.647855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.671591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.672189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.672462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.672735 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.673449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:12.673746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.675191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.692415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 557/1097] 10.222.0.20 () {56 vars in 1133 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:12.698029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.743827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 541/1098] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:12.749674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f0729-feff-49ca-b458-6baa022f9302 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.760733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.798647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558bc6b-4a67-4f25-8c37-daf77484f4b3 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.834550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545a8ee-450a-49c8-addc-6a23fb542805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.835150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a8ee-450a-49c8-addc-6a23fb542805 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.835430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a8ee-450a-49c8-addc-6a23fb542805 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.835695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545a8ee-450a-49c8-addc-6a23fb542805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.850308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f0729-feff-49ca-b458-6baa022f9302 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:12.852622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.884313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cabcc3-51cd-4fb3-b101-61d2445b9739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:12.884930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cabcc3-51cd-4fb3-b101-61d2445b9739 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:12.885206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cabcc3-51cd-4fb3-b101-61d2445b9739 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:12.885480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cabcc3-51cd-4fb3-b101-61d2445b9739 None admin] PATH_INFO: `/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:12.886646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cabcc3-51cd-4fb3-b101-61d2445b9739 None admin] RBAC: Authorizing `identity:get_project(project_id=ced8a31d51514742a0fc8f38b1d3f96b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.888120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cabcc3-51cd-4fb3-b101-61d2445b9739 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.889702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 542/1099] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:12.896801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cabcc3-51cd-4fb3-b101-61d2445b9739 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:12.902272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545a8ee-450a-49c8-addc-6a23fb542805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:12.903727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545a8ee-450a-49c8-addc-6a23fb542805 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:12.964337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:12.993979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cabcc3-51cd-4fb3-b101-61d2445b9739 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:13.002119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 558/1100] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 09 23:04:13.007851 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545a8ee-450a-49c8-addc-6a23fb542805 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:13.031504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:13.032106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:13.032382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:13.032652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:13.033400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=ced8a31d51514742a0fc8f38b1d3f96b, user.id=32e9f6f2673e4efaa26f98913e9ba77f) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:13.033703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:13.035325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:13.040919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 543/1101] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:12 2022] GET /identity/v3/role_assignments?user.id=32e9f6f2673e4efaa26f98913e9ba77f&scope.project.id=ced8a31d51514742a0fc8f38b1d3f96b&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:13.111279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545a8ee-450a-49c8-addc-6a23fb542805 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:13.147391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:13.147965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:13.148240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:13.148506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:13.159207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:13.161202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:04:13.176924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:13.178147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:04:13.179479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:04:13.179479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:04:13.179479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:13.179479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 559/1102] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:13.183880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:13.221394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:13.294933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e75e0ce-cf4c-49bf-a708-cd174b2d6b5c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:13.328081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:13.328675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:13.328947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:13.329208 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:13.329833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:13.330124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:13.332516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:13.343913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 544/1103] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:13.350213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:13.374002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:13.477518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415d1d9-942a-4fdc-9567-5a8237546205 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:13.519180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:13.520117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:13.520385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:13.522647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:13.532337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:13.534100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:04:13.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:13.552592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:13.555556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 560/1104] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:04:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:04:13.559384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:13.684879 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:13.694793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215ee79-f1dd-44a0-8452-5d850bd7deb1 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:13.739197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a054bd-81ce-45bf-baf3-5b21a5fd74e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:13.739855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a054bd-81ce-45bf-baf3-5b21a5fd74e4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:13.740137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a054bd-81ce-45bf-baf3-5b21a5fd74e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:13.740413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a054bd-81ce-45bf-baf3-5b21a5fd74e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:13.741117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a054bd-81ce-45bf-baf3-5b21a5fd74e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:13.741425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a054bd-81ce-45bf-baf3-5b21a5fd74e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:13.743260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a054bd-81ce-45bf-baf3-5b21a5fd74e4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:13.755710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 545/1105] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:04:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:13.762094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:13.844594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:13.872058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd702d-02eb-4087-bb37-2d70a172e6c7 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:13.910840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2657273-0516-43b7-b3e5-ede1014ddb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:13.911441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2657273-0516-43b7-b3e5-ede1014ddb32 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:13.911718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2657273-0516-43b7-b3e5-ede1014ddb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:13.911994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2657273-0516-43b7-b3e5-ede1014ddb32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:13.912724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2657273-0516-43b7-b3e5-ede1014ddb32 None admin] RBAC: Adding query filter params (role.id=a65ba9a33bc44775a27c2cfdd3a54a42, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=c3bec982c8474c349ca4901039844d26) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:13.913032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2657273-0516-43b7-b3e5-ede1014ddb32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:13.914481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2657273-0516-43b7-b3e5-ede1014ddb32 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:13.921412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 561/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:13 2022] GET /identity/v3/role_assignments?user.id=c3bec982c8474c349ca4901039844d26&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=a65ba9a33bc44775a27c2cfdd3a54a42 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:15.404071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f098536-3a5b-4ea8-b820-c73603988657 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:15.404628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f098536-3a5b-4ea8-b820-c73603988657 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:15.404628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f098536-3a5b-4ea8-b820-c73603988657 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:15.405974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 546/1107] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:15.411961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f96bd-0b30-4e4b-944e-e2a40460246b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:15.412411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f96bd-0b30-4e4b-944e-e2a40460246b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:15.412500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f96bd-0b30-4e4b-944e-e2a40460246b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:15.431522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ec4f96bd-0b30-4e4b-944e-e2a40460246b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:15.432697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ec4f96bd-0b30-4e4b-944e-e2a40460246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:15.510881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 562/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:15.792521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f098536-3a5b-4ea8-b820-c73603988657 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:15.892528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f098536-3a5b-4ea8-b820-c73603988657 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:15.924905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb144c-fbfa-44cd-bb6a-c6b2f797ed52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:15.926099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb144c-fbfa-44cd-bb6a-c6b2f797ed52 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:15.926405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb144c-fbfa-44cd-bb6a-c6b2f797ed52 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:15.926680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb144c-fbfa-44cd-bb6a-c6b2f797ed52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:15.927792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb144c-fbfa-44cd-bb6a-c6b2f797ed52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:15.929218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb144c-fbfa-44cd-bb6a-c6b2f797ed52 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:15.930824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 547/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:15.936665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f96bd-0b30-4e4b-944e-e2a40460246b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:16.002617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:16.031740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f96bd-0b30-4e4b-944e-e2a40460246b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:16.064712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f94d5-d8ea-4a5a-8c70-f06fcf5a2de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:16.065242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f94d5-d8ea-4a5a-8c70-f06fcf5a2de0 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:16.065518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f94d5-d8ea-4a5a-8c70-f06fcf5a2de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:16.066494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f94d5-d8ea-4a5a-8c70-f06fcf5a2de0 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:16.067306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f94d5-d8ea-4a5a-8c70-f06fcf5a2de0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:16.068729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f94d5-d8ea-4a5a-8c70-f06fcf5a2de0 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:16.163330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 563/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:04:15 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:18.320452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93760-2c51-44af-ac0b-1c41891041c7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:18.320985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93760-2c51-44af-ac0b-1c41891041c7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:18.320985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93760-2c51-44af-ac0b-1c41891041c7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:18.322326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 548/1111] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:18.327425 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead58c-3be1-4cd4-bbdd-976671827959 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:18.327819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead58c-3be1-4cd4-bbdd-976671827959 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:18.327963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ead58c-3be1-4cd4-bbdd-976671827959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:18.346545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a6ead58c-3be1-4cd4-bbdd-976671827959 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:18.348300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a6ead58c-3be1-4cd4-bbdd-976671827959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:18.412433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 564/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:18.678764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f93760-2c51-44af-ac0b-1c41891041c7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:18.703447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:18.779326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f93760-2c51-44af-ac0b-1c41891041c7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:18.812369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db39d7-6161-4380-974f-468230d66fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:18.812978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db39d7-6161-4380-974f-468230d66fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:18.813250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db39d7-6161-4380-974f-468230d66fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:18.813517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db39d7-6161-4380-974f-468230d66fa2 None admin] PATH_INFO: `/v3/roles/54e2dc31e78b4516b2296086e9eb12c4` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:18.814921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db39d7-6161-4380-974f-468230d66fa2 None admin] RBAC: Authorizing `identity:get_role(role_id=54e2dc31e78b4516b2296086e9eb12c4)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:18.816895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db39d7-6161-4380-974f-468230d66fa2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:18.818214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 549/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:18 2022] GET /identity/v3/roles/54e2dc31e78b4516b2296086e9eb12c4 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:18.824247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ead58c-3be1-4cd4-bbdd-976671827959 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:18.879302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:18.921284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ead58c-3be1-4cd4-bbdd-976671827959 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:18.953897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b93619-86d2-4874-98df-1583f41d178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:18.954474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93619-86d2-4874-98df-1583f41d178d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:18.954752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93619-86d2-4874-98df-1583f41d178d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:18.955021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b93619-86d2-4874-98df-1583f41d178d None admin] PATH_INFO: `/v3/users/a4c6257a60be4c25a13391fd5c8e509c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:18.969043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b93619-86d2-4874-98df-1583f41d178d None admin] RBAC: Authorizing `identity:get_user(user_id=a4c6257a60be4c25a13391fd5c8e509c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:18.972550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b93619-86d2-4874-98df-1583f41d178d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:18.972550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 565/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:18 2022] GET /identity/v3/users/a4c6257a60be4c25a13391fd5c8e509c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:18.977921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db39d7-6161-4380-974f-468230d66fa2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:19.067647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:19.075940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db39d7-6161-4380-974f-468230d66fa2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:19.110228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0fc41-7f37-4177-98ab-899bfaa7609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:19.110850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0fc41-7f37-4177-98ab-899bfaa7609e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:19.111123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0fc41-7f37-4177-98ab-899bfaa7609e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:19.111388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0fc41-7f37-4177-98ab-899bfaa7609e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:19.176262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0fc41-7f37-4177-98ab-899bfaa7609e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:19.177690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0fc41-7f37-4177-98ab-899bfaa7609e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:19.263622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:19.272788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 550/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:19.278731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b93619-86d2-4874-98df-1583f41d178d None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:19.375353 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b93619-86d2-4874-98df-1583f41d178d None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:19.393161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:19.413011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ef1f6-51cb-4d49-9d0b-53b897dc238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:19.413642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ef1f6-51cb-4d49-9d0b-53b897dc238b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:19.413917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ef1f6-51cb-4d49-9d0b-53b897dc238b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:19.414187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ef1f6-51cb-4d49-9d0b-53b897dc238b None admin] PATH_INFO: `/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:19.415330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ef1f6-51cb-4d49-9d0b-53b897dc238b None admin] RBAC: Authorizing `identity:get_project(project_id=ced8a31d51514742a0fc8f38b1d3f96b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:19.416748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ef1f6-51cb-4d49-9d0b-53b897dc238b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:19.418401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 566/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:19 2022] GET /identity/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:19.424601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0fc41-7f37-4177-98ab-899bfaa7609e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:19.522080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0fc41-7f37-4177-98ab-899bfaa7609e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:19.555426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f8f1f-d1d3-4a4f-94f8-f26c36acd797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:19.556016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8f1f-d1d3-4a4f-94f8-f26c36acd797 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:19.556293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8f1f-d1d3-4a4f-94f8-f26c36acd797 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:19.556564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f8f1f-d1d3-4a4f-94f8-f26c36acd797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:19.557267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8f1f-d1d3-4a4f-94f8-f26c36acd797 None admin] RBAC: Adding query filter params (role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=ced8a31d51514742a0fc8f38b1d3f96b, user.id=a4c6257a60be4c25a13391fd5c8e509c) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:19.557574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8f1f-d1d3-4a4f-94f8-f26c36acd797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:19.559015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f8f1f-d1d3-4a4f-94f8-f26c36acd797 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:19.564214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 551/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:19 2022] GET /identity/v3/role_assignments?user.id=a4c6257a60be4c25a13391fd5c8e509c&scope.project.id=ced8a31d51514742a0fc8f38b1d3f96b&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:21.683151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bfb9d-7bb7-45d7-9caa-23327a19b1c4 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:21.683671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bfb9d-7bb7-45d7-9caa-23327a19b1c4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:21.683671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bfb9d-7bb7-45d7-9caa-23327a19b1c4 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:21.684949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 567/1118] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:21.691661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da4d9-78e6-4bf0-9653-2e1576f0dab6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:21.692139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da4d9-78e6-4bf0-9653-2e1576f0dab6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:21.692616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15da4d9-78e6-4bf0-9653-2e1576f0dab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:21.718377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e15da4d9-78e6-4bf0-9653-2e1576f0dab6 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:21.719577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e15da4d9-78e6-4bf0-9653-2e1576f0dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:21.776668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:21.792658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 552/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:22.072154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bfb9d-7bb7-45d7-9caa-23327a19b1c4 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:22.171291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bfb9d-7bb7-45d7-9caa-23327a19b1c4 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:22.204380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cb532-c13f-40f9-bcff-9f225556feda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:22.204983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cb532-c13f-40f9-bcff-9f225556feda None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:22.205256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cb532-c13f-40f9-bcff-9f225556feda None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:22.205523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cb532-c13f-40f9-bcff-9f225556feda None admin] PATH_INFO: `/v3/roles/54e2dc31e78b4516b2296086e9eb12c4` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:22.207069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cb532-c13f-40f9-bcff-9f225556feda None admin] RBAC: Authorizing `identity:get_role(role_id=54e2dc31e78b4516b2296086e9eb12c4)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:22.208678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2cb532-c13f-40f9-bcff-9f225556feda None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:22.209964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 568/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:22 2022] GET /identity/v3/roles/54e2dc31e78b4516b2296086e9eb12c4 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:22.215717 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15da4d9-78e6-4bf0-9653-2e1576f0dab6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:22.311372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15da4d9-78e6-4bf0-9653-2e1576f0dab6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:22.353391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab90630-58f5-4c91-9857-1bcf029240f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:22.353939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab90630-58f5-4c91-9857-1bcf029240f4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:22.354209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab90630-58f5-4c91-9857-1bcf029240f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:22.354509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab90630-58f5-4c91-9857-1bcf029240f4 None admin] PATH_INFO: `/v3/users/a4c6257a60be4c25a13391fd5c8e509c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:22.355595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab90630-58f5-4c91-9857-1bcf029240f4 None admin] RBAC: Authorizing `identity:get_user(user_id=a4c6257a60be4c25a13391fd5c8e509c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:22.357000 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab90630-58f5-4c91-9857-1bcf029240f4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:22.358598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 553/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:22 2022] GET /identity/v3/users/a4c6257a60be4c25a13391fd5c8e509c => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:22.365494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cb532-c13f-40f9-bcff-9f225556feda None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:22.415354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:22.464948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cb532-c13f-40f9-bcff-9f225556feda None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:22.497890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7613508a-d3d3-4152-b760-27b441367d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:22.498529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613508a-d3d3-4152-b760-27b441367d0c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:22.498805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613508a-d3d3-4152-b760-27b441367d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:22.499076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613508a-d3d3-4152-b760-27b441367d0c None admin] PATH_INFO: `/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:22.500170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7613508a-d3d3-4152-b760-27b441367d0c None admin] RBAC: Authorizing `identity:get_project(project_id=ced8a31d51514742a0fc8f38b1d3f96b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:22.501589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7613508a-d3d3-4152-b760-27b441367d0c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:22.503723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 569/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:22 2022] GET /identity/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:22.509207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab90630-58f5-4c91-9857-1bcf029240f4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:22.562760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:22.606781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab90630-58f5-4c91-9857-1bcf029240f4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:22.639523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a4056-c554-4ed6-92a0-2e3b9006aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:22.640160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a4056-c554-4ed6-92a0-2e3b9006aea3 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:22.640437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a4056-c554-4ed6-92a0-2e3b9006aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:22.640707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a4056-c554-4ed6-92a0-2e3b9006aea3 None admin] PATH_INFO: `/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b/users/a4c6257a60be4c25a13391fd5c8e509c/roles/54e2dc31e78b4516b2296086e9eb12c4` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:22.642851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a4056-c554-4ed6-92a0-2e3b9006aea3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ced8a31d51514742a0fc8f38b1d3f96b, user_id=a4c6257a60be4c25a13391fd5c8e509c, role_id=54e2dc31e78b4516b2296086e9eb12c4)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:22.644395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a4056-c554-4ed6-92a0-2e3b9006aea3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:22.714819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 554/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:04:22 2022] PUT /identity/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b/users/a4c6257a60be4c25a13391fd5c8e509c/roles/54e2dc31e78b4516b2296086e9eb12c4 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:24.810447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376c6e-4854-42e0-8a56-6c6a6a437201 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:24.810965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376c6e-4854-42e0-8a56-6c6a6a437201 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:24.810965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3376c6e-4854-42e0-8a56-6c6a6a437201 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:24.812264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 570/1124] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:24.817512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f00d0-ab8a-4e2d-85e3-d4338cf4c896 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:24.817930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f00d0-ab8a-4e2d-85e3-d4338cf4c896 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:24.818044 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f00d0-ab8a-4e2d-85e3-d4338cf4c896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:24.837055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a63f00d0-ab8a-4e2d-85e3-d4338cf4c896 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:24.838225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a63f00d0-ab8a-4e2d-85e3-d4338cf4c896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:24.902474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 555/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:25.167923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3376c6e-4854-42e0-8a56-6c6a6a437201 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:25.267463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3376c6e-4854-42e0-8a56-6c6a6a437201 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:25.284275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:25.299945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd43d1-5c9c-4276-99fa-679ded787f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:25.305106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd43d1-5c9c-4276-99fa-679ded787f6b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:25.305106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd43d1-5c9c-4276-99fa-679ded787f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:25.305106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd43d1-5c9c-4276-99fa-679ded787f6b None admin] PATH_INFO: `/v3/roles/54e2dc31e78b4516b2296086e9eb12c4` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:25.305106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd43d1-5c9c-4276-99fa-679ded787f6b None admin] RBAC: Authorizing `identity:get_role(role_id=54e2dc31e78b4516b2296086e9eb12c4)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:25.305106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd43d1-5c9c-4276-99fa-679ded787f6b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:25.305744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 571/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:25 2022] GET /identity/v3/roles/54e2dc31e78b4516b2296086e9eb12c4 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:25.310863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f00d0-ab8a-4e2d-85e3-d4338cf4c896 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:25.355332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:25.407916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f00d0-ab8a-4e2d-85e3-d4338cf4c896 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:25.441285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e043fc-8956-42f5-ab5a-6e3185a85e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:25.441844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e043fc-8956-42f5-ab5a-6e3185a85e78 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:25.442115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e043fc-8956-42f5-ab5a-6e3185a85e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:25.442403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e043fc-8956-42f5-ab5a-6e3185a85e78 None admin] PATH_INFO: `/v3/users/a4c6257a60be4c25a13391fd5c8e509c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:25.443496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e043fc-8956-42f5-ab5a-6e3185a85e78 None admin] RBAC: Authorizing `identity:get_user(user_id=a4c6257a60be4c25a13391fd5c8e509c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:25.444907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e043fc-8956-42f5-ab5a-6e3185a85e78 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:25.446495 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 556/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:25 2022] GET /identity/v3/users/a4c6257a60be4c25a13391fd5c8e509c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:25.452246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd43d1-5c9c-4276-99fa-679ded787f6b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:25.548895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd43d1-5c9c-4276-99fa-679ded787f6b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:25.581811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a933dc3-81ba-431e-983b-4d8330463e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:25.582720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933dc3-81ba-431e-983b-4d8330463e1b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:25.582996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933dc3-81ba-431e-983b-4d8330463e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:25.583260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a933dc3-81ba-431e-983b-4d8330463e1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:25.634184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:25.647021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a933dc3-81ba-431e-983b-4d8330463e1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:25.648435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a933dc3-81ba-431e-983b-4d8330463e1b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:25.743134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 572/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:25.748981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e043fc-8956-42f5-ab5a-6e3185a85e78 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:25.847539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e043fc-8956-42f5-ab5a-6e3185a85e78 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:25.859512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:25.879919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a19341-9e36-4566-90be-eb21731ff3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:25.880533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19341-9e36-4566-90be-eb21731ff3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:25.880836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19341-9e36-4566-90be-eb21731ff3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:25.881081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19341-9e36-4566-90be-eb21731ff3c8 None admin] PATH_INFO: `/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:25.882186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a19341-9e36-4566-90be-eb21731ff3c8 None admin] RBAC: Authorizing `identity:get_project(project_id=ced8a31d51514742a0fc8f38b1d3f96b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:25.883634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a19341-9e36-4566-90be-eb21731ff3c8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:25.885262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 557/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:25 2022] GET /identity/v3/projects/ced8a31d51514742a0fc8f38b1d3f96b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:25.891596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a933dc3-81ba-431e-983b-4d8330463e1b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:25.972593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:25.986492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a933dc3-81ba-431e-983b-4d8330463e1b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:26.021023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727fa7a-0d94-4b05-935b-47f1e4ec8739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:26.021936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727fa7a-0d94-4b05-935b-47f1e4ec8739 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:26.022449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727fa7a-0d94-4b05-935b-47f1e4ec8739 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:26.022916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727fa7a-0d94-4b05-935b-47f1e4ec8739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:26.024202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727fa7a-0d94-4b05-935b-47f1e4ec8739 None admin] RBAC: Adding query filter params (role.id=54e2dc31e78b4516b2296086e9eb12c4, scope.project.id=ced8a31d51514742a0fc8f38b1d3f96b, user.id=a4c6257a60be4c25a13391fd5c8e509c) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:26.024518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727fa7a-0d94-4b05-935b-47f1e4ec8739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:26.025924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727fa7a-0d94-4b05-935b-47f1e4ec8739 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:26.031493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 573/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:25 2022] GET /identity/v3/role_assignments?user.id=a4c6257a60be4c25a13391fd5c8e509c&scope.project.id=ced8a31d51514742a0fc8f38b1d3f96b&role.id=54e2dc31e78b4516b2296086e9eb12c4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:28.228267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c0de2-4f2a-4141-bfb5-faf8d782c3b4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:28.228795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c0de2-4f2a-4141-bfb5-faf8d782c3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:28.228795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c0de2-4f2a-4141-bfb5-faf8d782c3b4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:28.230128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 558/1131] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:28.235386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f943f-940d-4337-b1b7-25530b253570 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:28.235762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f943f-940d-4337-b1b7-25530b253570 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:28.235942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f943f-940d-4337-b1b7-25530b253570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:28.254311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a21f943f-940d-4337-b1b7-25530b253570 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:28.255541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a21f943f-940d-4337-b1b7-25530b253570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:28.319791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 574/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:28.589432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c0de2-4f2a-4141-bfb5-faf8d782c3b4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:28.687055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c0de2-4f2a-4141-bfb5-faf8d782c3b4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:28.719654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960e5cb-f6f4-4c3e-b448-04ea1b55c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:28.727025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960e5cb-f6f4-4c3e-b448-04ea1b55c146 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:28.727025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960e5cb-f6f4-4c3e-b448-04ea1b55c146 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:28.727025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960e5cb-f6f4-4c3e-b448-04ea1b55c146 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:28.727025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960e5cb-f6f4-4c3e-b448-04ea1b55c146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:28.727025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960e5cb-f6f4-4c3e-b448-04ea1b55c146 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:28.727025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 559/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:28.730971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f943f-940d-4337-b1b7-25530b253570 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:28.809736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:28.827926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f943f-940d-4337-b1b7-25530b253570 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:28.860255 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684963d-51c4-4f44-83a8-6cbe176f3106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:28.860777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684963d-51c4-4f44-83a8-6cbe176f3106 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:28.861047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684963d-51c4-4f44-83a8-6cbe176f3106 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:28.861310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684963d-51c4-4f44-83a8-6cbe176f3106 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:28.861952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684963d-51c4-4f44-83a8-6cbe176f3106 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:28.863944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684963d-51c4-4f44-83a8-6cbe176f3106 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:28.949900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 575/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:04:28 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:31.133243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b3e3d-cc16-4972-8522-6d6bfd600009 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:31.133780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b3e3d-cc16-4972-8522-6d6bfd600009 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:31.133780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b3e3d-cc16-4972-8522-6d6bfd600009 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:31.135162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 560/1135] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:31.140699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4146170-d94b-42c3-a2ed-7c5f8abd7764 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:31.141159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4146170-d94b-42c3-a2ed-7c5f8abd7764 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:31.141531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4146170-d94b-42c3-a2ed-7c5f8abd7764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:31.163486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c4146170-d94b-42c3-a2ed-7c5f8abd7764 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:31.164677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c4146170-d94b-42c3-a2ed-7c5f8abd7764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:31.230328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 576/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 09 23:04:31.513125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b3e3d-cc16-4972-8522-6d6bfd600009 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:31.563003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:31.613751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b3e3d-cc16-4972-8522-6d6bfd600009 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:31.647264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd681b42-cbf4-4948-ba95-80e59cbf3589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:31.647885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd681b42-cbf4-4948-ba95-80e59cbf3589 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:31.648163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd681b42-cbf4-4948-ba95-80e59cbf3589 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:31.648432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd681b42-cbf4-4948-ba95-80e59cbf3589 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:31.649544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd681b42-cbf4-4948-ba95-80e59cbf3589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:31.650985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd681b42-cbf4-4948-ba95-80e59cbf3589 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:31.652550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 561/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:04:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:31.658260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4146170-d94b-42c3-a2ed-7c5f8abd7764 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:31.721274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:31.754958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4146170-d94b-42c3-a2ed-7c5f8abd7764 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:31.790006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29cc56-7317-4124-a83e-6ca6531886e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:31.790630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29cc56-7317-4124-a83e-6ca6531886e2 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:31.790900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29cc56-7317-4124-a83e-6ca6531886e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:31.791162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29cc56-7317-4124-a83e-6ca6531886e2 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:31.791808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29cc56-7317-4124-a83e-6ca6531886e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:31.793264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29cc56-7317-4124-a83e-6ca6531886e2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:31.905055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 577/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:04:31 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:34.139788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350b4e6-6d1d-4c82-a3b7-d28f0b3af697 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:34.140318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350b4e6-6d1d-4c82-a3b7-d28f0b3af697 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:34.140318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350b4e6-6d1d-4c82-a3b7-d28f0b3af697 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:34.141646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 562/1139] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:34.147344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720777-9406-4c91-a8f4-993376153da4 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:34.147763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720777-9406-4c91-a8f4-993376153da4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:34.147890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c720777-9406-4c91-a8f4-993376153da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:34.166174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2c720777-9406-4c91-a8f4-993376153da4 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:34.167387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2c720777-9406-4c91-a8f4-993376153da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:34.231535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 578/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:34.505369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350b4e6-6d1d-4c82-a3b7-d28f0b3af697 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:34.603820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350b4e6-6d1d-4c82-a3b7-d28f0b3af697 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:34.619975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:34.637151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c522b4f-bcc2-472c-9be5-fb8866026fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:34.637760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c522b4f-bcc2-472c-9be5-fb8866026fde None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:34.646807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c522b4f-bcc2-472c-9be5-fb8866026fde None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:34.646807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c522b4f-bcc2-472c-9be5-fb8866026fde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:34.647855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c522b4f-bcc2-472c-9be5-fb8866026fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-0c522b4f-bcc2-472c-9be5-fb8866026fde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:34.652523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:34.653602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:34.654869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 563/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:04:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:34.659681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c720777-9406-4c91-a8f4-993376153da4 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:34.706948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:34.757063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c720777-9406-4c91-a8f4-993376153da4 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:34.789544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:34.790127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:34.790420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:34.790685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:34.791323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:34.791616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:34.792943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:34.801273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 579/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:04:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:04:34.807433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c522b4f-bcc2-472c-9be5-fb8866026fde None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:34.919325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c522b4f-bcc2-472c-9be5-fb8866026fde None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:34.952363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb5d97-73fc-47a3-8f70-dd2694efe923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:34.952973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb5d97-73fc-47a3-8f70-dd2694efe923 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:34.953257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb5d97-73fc-47a3-8f70-dd2694efe923 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:34.953529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb5d97-73fc-47a3-8f70-dd2694efe923 None admin] PATH_INFO: `/v3/users/8df249c78b3e4ab391992d0ed59377c8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:34.966692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb5d97-73fc-47a3-8f70-dd2694efe923 None admin] RBAC: Authorizing `identity:get_user(user_id=8df249c78b3e4ab391992d0ed59377c8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:34.968069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb5d97-73fc-47a3-8f70-dd2694efe923 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:34.969667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 564/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:34 2022] GET /identity/v3/users/8df249c78b3e4ab391992d0ed59377c8 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:34.975990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:35.073173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00453e6-0d88-4999-a403-8fd9bd5ec736 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:35.083405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:35.106017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62d98-5b49-49ea-9f45-b76451683d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:35.106628 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62d98-5b49-49ea-9f45-b76451683d6a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:35.106901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62d98-5b49-49ea-9f45-b76451683d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:35.107165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc62d98-5b49-49ea-9f45-b76451683d6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:35.171305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc62d98-5b49-49ea-9f45-b76451683d6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:35.172704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc62d98-5b49-49ea-9f45-b76451683d6a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:35.265504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 580/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:35.271898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb5d97-73fc-47a3-8f70-dd2694efe923 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:35.304784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:35.367981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb5d97-73fc-47a3-8f70-dd2694efe923 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:35.400494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50426c08-0b50-4825-9743-cf18d7a7b8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:35.401104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426c08-0b50-4825-9743-cf18d7a7b8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:35.401386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426c08-0b50-4825-9743-cf18d7a7b8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:35.401656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50426c08-0b50-4825-9743-cf18d7a7b8c2 None admin] PATH_INFO: `/v3/projects/273498b3031c4d778060f9567b37d542` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:35.402783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50426c08-0b50-4825-9743-cf18d7a7b8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=273498b3031c4d778060f9567b37d542)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:35.404203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50426c08-0b50-4825-9743-cf18d7a7b8c2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:35.405816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 565/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:35 2022] GET /identity/v3/projects/273498b3031c4d778060f9567b37d542 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:35.412529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62d98-5b49-49ea-9f45-b76451683d6a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:35.425186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:35.508840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc62d98-5b49-49ea-9f45-b76451683d6a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:35.541809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84886de-8c37-4853-a03a-a94a44401fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:35.542332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84886de-8c37-4853-a03a-a94a44401fcc None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:35.542631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84886de-8c37-4853-a03a-a94a44401fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:35.542896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84886de-8c37-4853-a03a-a94a44401fcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:35.543596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84886de-8c37-4853-a03a-a94a44401fcc None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=273498b3031c4d778060f9567b37d542, user.id=8df249c78b3e4ab391992d0ed59377c8) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:35.543893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84886de-8c37-4853-a03a-a94a44401fcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:35.545276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84886de-8c37-4853-a03a-a94a44401fcc None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:35.550442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 581/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:35 2022] GET /identity/v3/role_assignments?user.id=8df249c78b3e4ab391992d0ed59377c8&scope.project.id=273498b3031c4d778060f9567b37d542&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:37.661061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b6feb-0976-4d2f-a901-900d64fae1f7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:37.661582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b6feb-0976-4d2f-a901-900d64fae1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:37.661582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b6feb-0976-4d2f-a901-900d64fae1f7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:37.662967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 566/1147] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:37.668015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ebea-fbde-4399-aa3c-d9ae59258d5a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:37.668435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ebea-fbde-4399-aa3c-d9ae59258d5a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:37.668528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42ebea-fbde-4399-aa3c-d9ae59258d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:37.686559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ad42ebea-fbde-4399-aa3c-d9ae59258d5a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:37.687730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ad42ebea-fbde-4399-aa3c-d9ae59258d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:37.754240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 582/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:38.022748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b6feb-0976-4d2f-a901-900d64fae1f7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:38.121284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b6feb-0976-4d2f-a901-900d64fae1f7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:38.122759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:38.153928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f148538-230d-4776-b031-c40fd1d2389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:38.154565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f148538-230d-4776-b031-c40fd1d2389e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.154842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f148538-230d-4776-b031-c40fd1d2389e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.155109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f148538-230d-4776-b031-c40fd1d2389e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.165959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f148538-230d-4776-b031-c40fd1d2389e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-2f148538-230d-4776-b031-c40fd1d2389e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:38.170715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:38.171791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:38.172805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 567/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:04:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:38.178448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42ebea-fbde-4399-aa3c-d9ae59258d5a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:38.181499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:38.275446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42ebea-fbde-4399-aa3c-d9ae59258d5a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:38.308295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:38.308887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.309162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.309430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.310076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:38.310395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:38.311733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:38.319825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 583/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:04:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:38.325564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f148538-230d-4776-b031-c40fd1d2389e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:38.423592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f148538-230d-4776-b031-c40fd1d2389e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:38.460947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f678df9-78aa-41dd-8ebc-09dba58ec379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:38.461564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678df9-78aa-41dd-8ebc-09dba58ec379 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.461836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678df9-78aa-41dd-8ebc-09dba58ec379 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.462101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f678df9-78aa-41dd-8ebc-09dba58ec379 None admin] PATH_INFO: `/v3/users/8df249c78b3e4ab391992d0ed59377c8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.463245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f678df9-78aa-41dd-8ebc-09dba58ec379 None admin] RBAC: Authorizing `identity:get_user(user_id=8df249c78b3e4ab391992d0ed59377c8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:38.464659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f678df9-78aa-41dd-8ebc-09dba58ec379 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:38.466232 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 568/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:38 2022] GET /identity/v3/users/8df249c78b3e4ab391992d0ed59377c8 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:38.472724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:38.536103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:38.569109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c31d8-3b95-4396-8985-81437079d3ef None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:38.602854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4f970-27d7-4185-ae94-ca17265f7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:38.603472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4f970-27d7-4185-ae94-ca17265f7481 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.603748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4f970-27d7-4185-ae94-ca17265f7481 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.604018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4f970-27d7-4185-ae94-ca17265f7481 None admin] PATH_INFO: `/v3/projects/273498b3031c4d778060f9567b37d542` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.605116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4f970-27d7-4185-ae94-ca17265f7481 None admin] RBAC: Authorizing `identity:get_project(project_id=273498b3031c4d778060f9567b37d542)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:38.606623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4f970-27d7-4185-ae94-ca17265f7481 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:38.608242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 584/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:38 2022] GET /identity/v3/projects/273498b3031c4d778060f9567b37d542 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:38.614176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f678df9-78aa-41dd-8ebc-09dba58ec379 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:38.644679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:38.714031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f678df9-78aa-41dd-8ebc-09dba58ec379 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:38.748224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ff127-d6fb-4013-add4-91f7c09f61e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:38.748867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff127-d6fb-4013-add4-91f7c09f61e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:38.749148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff127-d6fb-4013-add4-91f7c09f61e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:38.749420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff127-d6fb-4013-add4-91f7c09f61e4 None admin] PATH_INFO: `/v3/projects/273498b3031c4d778060f9567b37d542/users/8df249c78b3e4ab391992d0ed59377c8/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:38.751394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ff127-d6fb-4013-add4-91f7c09f61e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=273498b3031c4d778060f9567b37d542, user_id=8df249c78b3e4ab391992d0ed59377c8, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:38.752797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283ff127-d6fb-4013-add4-91f7c09f61e4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:38.829114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 569/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:04:38 2022] PUT /identity/v3/projects/273498b3031c4d778060f9567b37d542/users/8df249c78b3e4ab391992d0ed59377c8/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:04:40.953922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdbf6-8016-4caa-85ca-41f9db57fecb None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:40.954501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdbf6-8016-4caa-85ca-41f9db57fecb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:40.954501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecdbf6-8016-4caa-85ca-41f9db57fecb None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:40.955819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 585/1154] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:40.961834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ce2b6-12f3-452c-8a5f-534a3e59ff75 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:40.962262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ce2b6-12f3-452c-8a5f-534a3e59ff75 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:40.962403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ce2b6-12f3-452c-8a5f-534a3e59ff75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:40.979950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b94ce2b6-12f3-452c-8a5f-534a3e59ff75 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:40.981110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b94ce2b6-12f3-452c-8a5f-534a3e59ff75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:41.044401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 570/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:41.309150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecdbf6-8016-4caa-85ca-41f9db57fecb None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:41.407776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecdbf6-8016-4caa-85ca-41f9db57fecb None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:41.442114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3402f2d-9cf2-40e1-9ff0-03c05c1f44ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:41.442748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3402f2d-9cf2-40e1-9ff0-03c05c1f44ed None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:41.443023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3402f2d-9cf2-40e1-9ff0-03c05c1f44ed None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:41.443287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3402f2d-9cf2-40e1-9ff0-03c05c1f44ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:41.445099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:41.453430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3402f2d-9cf2-40e1-9ff0-03c05c1f44ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-e3402f2d-9cf2-40e1-9ff0-03c05c1f44ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:41.458877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:41.475188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:41.476231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 586/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:04:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:41.477090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ce2b6-12f3-452c-8a5f-534a3e59ff75 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:41.486720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:41.561515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ce2b6-12f3-452c-8a5f-534a3e59ff75 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:41.594220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:41.594843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:41.595117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:41.595383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:41.596026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:41.596320 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:41.598224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:41.605951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 571/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:04:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:41.611877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3402f2d-9cf2-40e1-9ff0-03c05c1f44ed None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:41.706260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3402f2d-9cf2-40e1-9ff0-03c05c1f44ed None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:41.739376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd298-5346-4537-a0ca-b8f0d1bfe5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:41.739975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd298-5346-4537-a0ca-b8f0d1bfe5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:41.740246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd298-5346-4537-a0ca-b8f0d1bfe5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:41.740510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bd298-5346-4537-a0ca-b8f0d1bfe5a9 None admin] PATH_INFO: `/v3/users/8df249c78b3e4ab391992d0ed59377c8` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:41.741588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bd298-5346-4537-a0ca-b8f0d1bfe5a9 None admin] RBAC: Authorizing `identity:get_user(user_id=8df249c78b3e4ab391992d0ed59377c8)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:41.743015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bd298-5346-4537-a0ca-b8f0d1bfe5a9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:41.744551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 587/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:41 2022] GET /identity/v3/users/8df249c78b3e4ab391992d0ed59377c8 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:41.758484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:41.833067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:41.850970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daa492-95bf-49b0-978a-35eeaf4831b4 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:41.884872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403ce0-24b4-4e9b-bad9-f3ebd9250fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:41.885471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403ce0-24b4-4e9b-bad9-f3ebd9250fde None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:41.885750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403ce0-24b4-4e9b-bad9-f3ebd9250fde None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:41.886019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47403ce0-24b4-4e9b-bad9-f3ebd9250fde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:41.952476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403ce0-24b4-4e9b-bad9-f3ebd9250fde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:41.953943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47403ce0-24b4-4e9b-bad9-f3ebd9250fde None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:42.030553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:42.048257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 572/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:42.054819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd298-5346-4537-a0ca-b8f0d1bfe5a9 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:42.115034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:42.152896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bd298-5346-4537-a0ca-b8f0d1bfe5a9 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:42.185265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a402a3e-d9da-4cc0-bd3a-deb20b776081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:42.185876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402a3e-d9da-4cc0-bd3a-deb20b776081 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:42.186154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402a3e-d9da-4cc0-bd3a-deb20b776081 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:42.186451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a402a3e-d9da-4cc0-bd3a-deb20b776081 None admin] PATH_INFO: `/v3/projects/273498b3031c4d778060f9567b37d542` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:42.187551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a402a3e-d9da-4cc0-bd3a-deb20b776081 None admin] RBAC: Authorizing `identity:get_project(project_id=273498b3031c4d778060f9567b37d542)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:42.188971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a402a3e-d9da-4cc0-bd3a-deb20b776081 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:42.190618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 588/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:42 2022] GET /identity/v3/projects/273498b3031c4d778060f9567b37d542 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:42.197149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403ce0-24b4-4e9b-bad9-f3ebd9250fde None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:42.292298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47403ce0-24b4-4e9b-bad9-f3ebd9250fde None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:42.323999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6040c39-1a0a-46aa-9d37-ba96a4997927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:42.324579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040c39-1a0a-46aa-9d37-ba96a4997927 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:42.324852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040c39-1a0a-46aa-9d37-ba96a4997927 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:42.325118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040c39-1a0a-46aa-9d37-ba96a4997927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:42.325818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6040c39-1a0a-46aa-9d37-ba96a4997927 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=273498b3031c4d778060f9567b37d542, user.id=8df249c78b3e4ab391992d0ed59377c8) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:42.326116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6040c39-1a0a-46aa-9d37-ba96a4997927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:42.327539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6040c39-1a0a-46aa-9d37-ba96a4997927 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:42.333863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 573/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:42 2022] GET /identity/v3/role_assignments?user.id=8df249c78b3e4ab391992d0ed59377c8&scope.project.id=273498b3031c4d778060f9567b37d542&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:44.525784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ccd8d-ff0b-44ea-86e9-8b0cc5e716f5 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:44.526331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ccd8d-ff0b-44ea-86e9-8b0cc5e716f5 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:44.526331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ccd8d-ff0b-44ea-86e9-8b0cc5e716f5 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:44.527844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 589/1162] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:44.532996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532acdc-a8e2-4a49-8f46-24fb35f81388 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:44.533448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532acdc-a8e2-4a49-8f46-24fb35f81388 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:44.533544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532acdc-a8e2-4a49-8f46-24fb35f81388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:44.551653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c532acdc-a8e2-4a49-8f46-24fb35f81388 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:44.552824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c532acdc-a8e2-4a49-8f46-24fb35f81388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:44.592971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:44.616846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 574/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:44.889344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ccd8d-ff0b-44ea-86e9-8b0cc5e716f5 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:44.989809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ccd8d-ff0b-44ea-86e9-8b0cc5e716f5 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:45.017839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:45.024007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfd8f4-8f02-4a46-9ebc-fe2e8bbf5988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:45.024593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd8f4-8f02-4a46-9ebc-fe2e8bbf5988 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:45.024867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd8f4-8f02-4a46-9ebc-fe2e8bbf5988 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:45.025134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfd8f4-8f02-4a46-9ebc-fe2e8bbf5988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:45.090254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfd8f4-8f02-4a46-9ebc-fe2e8bbf5988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:45.091701 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfd8f4-8f02-4a46-9ebc-fe2e8bbf5988 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:45.185755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 590/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:45.192107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532acdc-a8e2-4a49-8f46-24fb35f81388 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:45.287915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532acdc-a8e2-4a49-8f46-24fb35f81388 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:45.320153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d7ab4-62fe-4f9c-a754-2fcd7c7ace54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:45.320795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d7ab4-62fe-4f9c-a754-2fcd7c7ace54 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:45.321069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d7ab4-62fe-4f9c-a754-2fcd7c7ace54 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:45.321333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d7ab4-62fe-4f9c-a754-2fcd7c7ace54 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:45.327747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d7ab4-62fe-4f9c-a754-2fcd7c7ace54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:45.329166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d7ab4-62fe-4f9c-a754-2fcd7c7ace54 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-bf9d7ab4-62fe-4f9c-a754-2fcd7c7ace54 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:45.340937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:45.342199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:04:45.343377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 575/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:04:45 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:45.348513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfd8f4-8f02-4a46-9ebc-fe2e8bbf5988 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:45.376871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:45.444474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfd8f4-8f02-4a46-9ebc-fe2e8bbf5988 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:45.477103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6aeb9-a146-4c43-9a97-17b95bfbe1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:45.477691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6aeb9-a146-4c43-9a97-17b95bfbe1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:45.477963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6aeb9-a146-4c43-9a97-17b95bfbe1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:45.478227 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6aeb9-a146-4c43-9a97-17b95bfbe1b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:45.478878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6aeb9-a146-4c43-9a97-17b95bfbe1b2 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:45.479171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6aeb9-a146-4c43-9a97-17b95bfbe1b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:45.480505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6aeb9-a146-4c43-9a97-17b95bfbe1b2 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:45.487337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 591/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Fri Sep 9 23:04:45 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:47.748467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121ce5-d970-41a1-9578-6248a1dc2c5d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:47.748990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121ce5-d970-41a1-9578-6248a1dc2c5d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:47.748990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6121ce5-d970-41a1-9578-6248a1dc2c5d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:47.750369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 576/1167] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:47.755543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70e168-6e3f-4ecc-bc8b-c4e74b2a5bd5 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:47.755947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70e168-6e3f-4ecc-bc8b-c4e74b2a5bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:47.756078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70e168-6e3f-4ecc-bc8b-c4e74b2a5bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:47.774075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-aa70e168-6e3f-4ecc-bc8b-c4e74b2a5bd5 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:47.775273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-aa70e168-6e3f-4ecc-bc8b-c4e74b2a5bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:47.840183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 592/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:48.115358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6121ce5-d970-41a1-9578-6248a1dc2c5d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:48.151128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:48.214554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6121ce5-d970-41a1-9578-6248a1dc2c5d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:48.248316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddd6d2-7a00-42ea-a6c3-417a77deca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:48.248902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddd6d2-7a00-42ea-a6c3-417a77deca16 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:48.249179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddd6d2-7a00-42ea-a6c3-417a77deca16 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:48.249450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddd6d2-7a00-42ea-a6c3-417a77deca16 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:48.250097 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddd6d2-7a00-42ea-a6c3-417a77deca16 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:48.251478 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddd6d2-7a00-42ea-a6c3-417a77deca16 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:48.323996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 577/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Fri Sep 9 23:04:48 2022] POST /identity/v3/domains => generated 267 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:04:50.493861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5080dd-114c-451a-b99d-51fa26784646 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:50.494446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5080dd-114c-451a-b99d-51fa26784646 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:50.494446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5080dd-114c-451a-b99d-51fa26784646 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:50.495777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 593/1170] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:50.501136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034103-0974-487e-a123-bebde4d700c3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:50.501565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034103-0974-487e-a123-bebde4d700c3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:50.501658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d034103-0974-487e-a123-bebde4d700c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:50.520481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3d034103-0974-487e-a123-bebde4d700c3 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:50.521692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3d034103-0974-487e-a123-bebde4d700c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:50.586190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 578/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:50.853078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5080dd-114c-451a-b99d-51fa26784646 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:50.877098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:50.950909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5080dd-114c-451a-b99d-51fa26784646 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:50.985136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708444e-90c7-4f16-893e-18f0dabee5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:50.985750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708444e-90c7-4f16-893e-18f0dabee5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:50.986028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708444e-90c7-4f16-893e-18f0dabee5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:50.986298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708444e-90c7-4f16-893e-18f0dabee5c6 None admin] PATH_INFO: `/v3/domains/e11d9b308dff4e1794bc2e422da5a0e1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:50.997138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708444e-90c7-4f16-893e-18f0dabee5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e11d9b308dff4e1794bc2e422da5a0e1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:50.998609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3708444e-90c7-4f16-893e-18f0dabee5c6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:51.000187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 594/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Fri Sep 9 23:04:50 2022] GET /identity/v3/domains/e11d9b308dff4e1794bc2e422da5a0e1 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:51.006567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d034103-0974-487e-a123-bebde4d700c3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:51.032312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:51.103023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d034103-0974-487e-a123-bebde4d700c3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:51.136466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f42e7-ceb8-4cb4-ad1d-2ff09bb32944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:51.137002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f42e7-ceb8-4cb4-ad1d-2ff09bb32944 None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:51.137280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f42e7-ceb8-4cb4-ad1d-2ff09bb32944 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:51.137551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f42e7-ceb8-4cb4-ad1d-2ff09bb32944 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:51.138189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f42e7-ceb8-4cb4-ad1d-2ff09bb32944 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:51.139635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f42e7-ceb8-4cb4-ad1d-2ff09bb32944 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:51.252692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 579/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:04:51 2022] POST /identity/v3/projects => generated 361 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:53.481373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff409c-0485-4afb-a7b5-cebf90610f5b None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.481840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff409c-0485-4afb-a7b5-cebf90610f5b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.481930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff409c-0485-4afb-a7b5-cebf90610f5b None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.483273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 595/1174] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:53.489078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f6c2-56a2-47e4-9d1a-e3d02e0076ef None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.489590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f6c2-56a2-47e4-9d1a-e3d02e0076ef None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.489590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0f6c2-56a2-47e4-9d1a-e3d02e0076ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.507806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4dc0f6c2-56a2-47e4-9d1a-e3d02e0076ef None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:53.508993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4dc0f6c2-56a2-47e4-9d1a-e3d02e0076ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:53.572120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 580/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:53.851282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff409c-0485-4afb-a7b5-cebf90610f5b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:53.933559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:53.949525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff409c-0485-4afb-a7b5-cebf90610f5b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:53.984141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa1185-e117-4bb8-a9ea-7bf798cddfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:53.984749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1185-e117-4bb8-a9ea-7bf798cddfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:53.985022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1185-e117-4bb8-a9ea-7bf798cddfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:53.985288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa1185-e117-4bb8-a9ea-7bf798cddfc6 None admin] PATH_INFO: `/v3/domains/e11d9b308dff4e1794bc2e422da5a0e1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:53.986392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa1185-e117-4bb8-a9ea-7bf798cddfc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e11d9b308dff4e1794bc2e422da5a0e1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:53.987822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa1185-e117-4bb8-a9ea-7bf798cddfc6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:53.989903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 596/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Fri Sep 9 23:04:53 2022] GET /identity/v3/domains/e11d9b308dff4e1794bc2e422da5a0e1 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:53.995648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0f6c2-56a2-47e4-9d1a-e3d02e0076ef None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:54.006141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:54.091074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0f6c2-56a2-47e4-9d1a-e3d02e0076ef None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:54.123350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda612e7-6977-4817-a872-d77d67e9977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:54.123880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda612e7-6977-4817-a872-d77d67e9977d None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:54.124151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda612e7-6977-4817-a872-d77d67e9977d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:54.124416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda612e7-6977-4817-a872-d77d67e9977d None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:54.125055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda612e7-6977-4817-a872-d77d67e9977d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:54.126478 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda612e7-6977-4817-a872-d77d67e9977d None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:54.197766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 581/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:04:53 2022] POST /identity/v3/users => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:04:56.421899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b8d3-76d5-4265-9c9b-8ec688e6262c None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:56.422291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b8d3-76d5-4265-9c9b-8ec688e6262c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:56.422459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b8d3-76d5-4265-9c9b-8ec688e6262c None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:56.423816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 597/1178] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:56.429661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dc310-0730-41cb-9006-1fd4e9fc3d33 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:56.430072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dc310-0730-41cb-9006-1fd4e9fc3d33 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:56.430203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9dc310-0730-41cb-9006-1fd4e9fc3d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:56.448804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ea9dc310-0730-41cb-9006-1fd4e9fc3d33 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:56.449981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ea9dc310-0730-41cb-9006-1fd4e9fc3d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:56.507954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:56.515988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 582/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:04:56.790595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5b8d3-76d5-4265-9c9b-8ec688e6262c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:56.889461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5b8d3-76d5-4265-9c9b-8ec688e6262c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:56.922354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af959476-7298-4156-a995-ed63ef58b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:56.922971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af959476-7298-4156-a995-ed63ef58b01b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:56.923245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af959476-7298-4156-a995-ed63ef58b01b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:56.923511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af959476-7298-4156-a995-ed63ef58b01b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:56.933449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af959476-7298-4156-a995-ed63ef58b01b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-af959476-7298-4156-a995-ed63ef58b01b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:04:56.938181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:04:56.939295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:04:56.940534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 598/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:04:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:04:56.945870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9dc310-0730-41cb-9006-1fd4e9fc3d33 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:57.044208 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9dc310-0730-41cb-9006-1fd4e9fc3d33 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:57.078553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:57.079145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:57.079418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:57.079682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:57.080328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:57.080630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:57.082014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:57.090210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 583/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:04:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:57.096161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af959476-7298-4156-a995-ed63ef58b01b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:57.110998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:57.193554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af959476-7298-4156-a995-ed63ef58b01b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:57.225888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beab1da-00c6-434a-9dbd-426429939365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:57.226522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab1da-00c6-434a-9dbd-426429939365 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:57.226796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab1da-00c6-434a-9dbd-426429939365 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:57.227064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab1da-00c6-434a-9dbd-426429939365 None admin] PATH_INFO: `/v3/users/75b229be4de4495e91ff73db474c9c90` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:57.240107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beab1da-00c6-434a-9dbd-426429939365 None admin] RBAC: Authorizing `identity:get_user(user_id=75b229be4de4495e91ff73db474c9c90)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:57.241520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beab1da-00c6-434a-9dbd-426429939365 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:57.243131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 599/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:04:57 2022] GET /identity/v3/users/75b229be4de4495e91ff73db474c9c90 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:57.248965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:57.303607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:57.345863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6820c-92d7-4be5-9930-ca4fb6d6667c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:57.378882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4524855-b3b0-4ca5-9602-9099d6074a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:57.379618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4524855-b3b0-4ca5-9602-9099d6074a78 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:57.379898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4524855-b3b0-4ca5-9602-9099d6074a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:57.380170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4524855-b3b0-4ca5-9602-9099d6074a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:57.445222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4524855-b3b0-4ca5-9602-9099d6074a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:57.446677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4524855-b3b0-4ca5-9602-9099d6074a78 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:57.499147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:57.542266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 584/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:04:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:04:57.548125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beab1da-00c6-434a-9dbd-426429939365 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:57.614781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:04:57.644243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beab1da-00c6-434a-9dbd-426429939365 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:57.677342 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb518b0b-8df9-4525-a221-a221a0471d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:57.677944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb518b0b-8df9-4525-a221-a221a0471d65 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:57.678220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb518b0b-8df9-4525-a221-a221a0471d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:57.678515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb518b0b-8df9-4525-a221-a221a0471d65 None admin] PATH_INFO: `/v3/projects/a4d007544bbc44e3935ffee978cb670f` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:57.679602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb518b0b-8df9-4525-a221-a221a0471d65 None admin] RBAC: Authorizing `identity:get_project(project_id=a4d007544bbc44e3935ffee978cb670f)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:57.681008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb518b0b-8df9-4525-a221-a221a0471d65 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:57.682632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 600/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:04:57 2022] GET /identity/v3/projects/a4d007544bbc44e3935ffee978cb670f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:57.688698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4524855-b3b0-4ca5-9602-9099d6074a78 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:04:57.783944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4524855-b3b0-4ca5-9602-9099d6074a78 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:04:57.816667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193118b-d43b-432a-9da5-1701d0426c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:04:57.817542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193118b-d43b-432a-9da5-1701d0426c00 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:57.817814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193118b-d43b-432a-9da5-1701d0426c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:57.818082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193118b-d43b-432a-9da5-1701d0426c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:57.818817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193118b-d43b-432a-9da5-1701d0426c00 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=a4d007544bbc44e3935ffee978cb670f, user.id=75b229be4de4495e91ff73db474c9c90) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:04:57.819118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193118b-d43b-432a-9da5-1701d0426c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:04:57.820528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193118b-d43b-432a-9da5-1701d0426c00 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:04:57.825755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 585/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:04:57 2022] GET /identity/v3/role_assignments?user.id=75b229be4de4495e91ff73db474c9c90&scope.project.id=a4d007544bbc44e3935ffee978cb670f&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:04:59.919452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5265e-17a2-469c-b852-bcd3d1774fc0 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:59.920007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5265e-17a2-469c-b852-bcd3d1774fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:59.920007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e5265e-17a2-469c-b852-bcd3d1774fc0 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:59.921310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 601/1186] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:04:59.926466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26147bd-6deb-4317-b687-0b78b1185fc7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:04:59.926897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26147bd-6deb-4317-b687-0b78b1185fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:04:59.927005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26147bd-6deb-4317-b687-0b78b1185fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:04:59.945043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a26147bd-6deb-4317-b687-0b78b1185fc7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:04:59.946266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a26147bd-6deb-4317-b687-0b78b1185fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:04:59.960530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:00.011625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 586/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:04:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:05:00.277740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5265e-17a2-469c-b852-bcd3d1774fc0 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:00.377339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e5265e-17a2-469c-b852-bcd3d1774fc0 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:00.411235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ecde4-2ce7-4358-a810-e8291722f746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:00.411860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ecde4-2ce7-4358-a810-e8291722f746 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:00.412138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ecde4-2ce7-4358-a810-e8291722f746 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:00.412408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ecde4-2ce7-4358-a810-e8291722f746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:00.419899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:00.430944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ecde4-2ce7-4358-a810-e8291722f746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-085ecde4-2ce7-4358-a810-e8291722f746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:05:00.435820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:05:00.436924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:05:00.438082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 602/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:05:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:05:00.443110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26147bd-6deb-4317-b687-0b78b1185fc7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:00.538953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26147bd-6deb-4317-b687-0b78b1185fc7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:00.571553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:00.572146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:00.572418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:00.572683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:00.573325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:05:00.573617 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:00.574990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:00.583532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 587/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:05:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:05:00.589467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ecde4-2ce7-4358-a810-e8291722f746 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:00.687781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ecde4-2ce7-4358-a810-e8291722f746 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:00.720508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d921525-4dc0-43c1-9816-bdf6a142d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:00.721164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d921525-4dc0-43c1-9816-bdf6a142d1df None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:00.721433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d921525-4dc0-43c1-9816-bdf6a142d1df None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:00.721696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d921525-4dc0-43c1-9816-bdf6a142d1df None admin] PATH_INFO: `/v3/users/75b229be4de4495e91ff73db474c9c90` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:00.723551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d921525-4dc0-43c1-9816-bdf6a142d1df None admin] RBAC: Authorizing `identity:get_user(user_id=75b229be4de4495e91ff73db474c9c90)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:00.725013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d921525-4dc0-43c1-9816-bdf6a142d1df None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:00.726577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 603/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:05:00 2022] GET /identity/v3/users/75b229be4de4495e91ff73db474c9c90 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:05:00.732553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:00.747953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:00.829049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bc3d5-8f27-4f14-9241-12a6f158d505 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:00.862534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a7c01-3353-46bf-9fb3-dd82daa14d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:00.863173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7c01-3353-46bf-9fb3-dd82daa14d3c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:00.863537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7c01-3353-46bf-9fb3-dd82daa14d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:00.863694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a7c01-3353-46bf-9fb3-dd82daa14d3c None admin] PATH_INFO: `/v3/projects/a4d007544bbc44e3935ffee978cb670f` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:00.864786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a7c01-3353-46bf-9fb3-dd82daa14d3c None admin] RBAC: Authorizing `identity:get_project(project_id=a4d007544bbc44e3935ffee978cb670f)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:00.866182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a7c01-3353-46bf-9fb3-dd82daa14d3c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:00.867816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 588/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:05:00 2022] GET /identity/v3/projects/a4d007544bbc44e3935ffee978cb670f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:05:00.873784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d921525-4dc0-43c1-9816-bdf6a142d1df None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:00.937431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:00.970795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d921525-4dc0-43c1-9816-bdf6a142d1df None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:01.003195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05f9d4-fe48-4906-96ca-9748b46676be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:01.003834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f9d4-fe48-4906-96ca-9748b46676be None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:01.004113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f9d4-fe48-4906-96ca-9748b46676be None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:01.004381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f9d4-fe48-4906-96ca-9748b46676be None admin] PATH_INFO: `/v3/projects/a4d007544bbc44e3935ffee978cb670f/users/75b229be4de4495e91ff73db474c9c90/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:01.006250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05f9d4-fe48-4906-96ca-9748b46676be None admin] RBAC: Authorizing `identity:create_grant(project_id=a4d007544bbc44e3935ffee978cb670f, user_id=75b229be4de4495e91ff73db474c9c90, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:01.007704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05f9d4-fe48-4906-96ca-9748b46676be None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:01.074612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 604/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:05:00 2022] PUT /identity/v3/projects/a4d007544bbc44e3935ffee978cb670f/users/75b229be4de4495e91ff73db474c9c90/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:05:03.230192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97e482-b844-4e54-b14d-37a3dff25854 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:03.230786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97e482-b844-4e54-b14d-37a3dff25854 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:03.230786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97e482-b844-4e54-b14d-37a3dff25854 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:03.232081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 589/1193] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:05:03.237259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368459b-5fb3-4945-8d4c-c7a2abfba63b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:03.237662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368459b-5fb3-4945-8d4c-c7a2abfba63b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:03.237838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368459b-5fb3-4945-8d4c-c7a2abfba63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:03.256571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4368459b-5fb3-4945-8d4c-c7a2abfba63b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:05:03.257750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4368459b-5fb3-4945-8d4c-c7a2abfba63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:05:03.326326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 605/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:05:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:05:03.600751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97e482-b844-4e54-b14d-37a3dff25854 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:03.679457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:03.698258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97e482-b844-4e54-b14d-37a3dff25854 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:03.731190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34f7e-6b9f-40e9-ae63-2f1e78398b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:03.731800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34f7e-6b9f-40e9-ae63-2f1e78398b6d None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:03.732104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34f7e-6b9f-40e9-ae63-2f1e78398b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:03.732343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34f7e-6b9f-40e9-ae63-2f1e78398b6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:03.742639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34f7e-6b9f-40e9-ae63-2f1e78398b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-42f34f7e-6b9f-40e9-ae63-2f1e78398b6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:05:03.747384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:05:03.748525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:05:03.749828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 590/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:05:03.755286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368459b-5fb3-4945-8d4c-c7a2abfba63b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:03.809061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:03.852349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368459b-5fb3-4945-8d4c-c7a2abfba63b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:03.886782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:03.887382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:03.887659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:03.887930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:03.888583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:05:03.888883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:03.890235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:03.898091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 606/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:05:03.904096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34f7e-6b9f-40e9-ae63-2f1e78398b6d None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:04.000590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34f7e-6b9f-40e9-ae63-2f1e78398b6d None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:04.033215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461c314-422c-476b-934c-240904e22ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:04.033817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c314-422c-476b-934c-240904e22ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:04.034092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c314-422c-476b-934c-240904e22ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:04.034377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461c314-422c-476b-934c-240904e22ab2 None admin] PATH_INFO: `/v3/users/75b229be4de4495e91ff73db474c9c90` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:04.035531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461c314-422c-476b-934c-240904e22ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=75b229be4de4495e91ff73db474c9c90)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:04.036939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461c314-422c-476b-934c-240904e22ab2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:04.038521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 591/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Fri Sep 9 23:05:03 2022] GET /identity/v3/users/75b229be4de4495e91ff73db474c9c90 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:05:04.044345 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:04.141650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807212b5-af9d-4a6a-b04b-1c049b3d519a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:04.157578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:04.174130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc4313-446c-442d-a693-5f71146656cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:04.174749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc4313-446c-442d-a693-5f71146656cc None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:04.175021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc4313-446c-442d-a693-5f71146656cc None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:04.175285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc4313-446c-442d-a693-5f71146656cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:04.240668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc4313-446c-442d-a693-5f71146656cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:04.242092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc4313-446c-442d-a693-5f71146656cc None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:04.336515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 607/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:05:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:05:04.342563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461c314-422c-476b-934c-240904e22ab2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:04.353480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:04.439471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461c314-422c-476b-934c-240904e22ab2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:04.472940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f831f20-b422-409b-b2ec-52858df094e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:04.473559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f831f20-b422-409b-b2ec-52858df094e9 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:04.473837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f831f20-b422-409b-b2ec-52858df094e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:04.474110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f831f20-b422-409b-b2ec-52858df094e9 None admin] PATH_INFO: `/v3/projects/a4d007544bbc44e3935ffee978cb670f` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:04.475252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f831f20-b422-409b-b2ec-52858df094e9 None admin] RBAC: Authorizing `identity:get_project(project_id=a4d007544bbc44e3935ffee978cb670f)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:04.476676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f831f20-b422-409b-b2ec-52858df094e9 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:04.478310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 592/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Fri Sep 9 23:05:04 2022] GET /identity/v3/projects/a4d007544bbc44e3935ffee978cb670f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:05:04.484450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc4313-446c-442d-a693-5f71146656cc None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:05:04.502329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:04.585263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc4313-446c-442d-a693-5f71146656cc None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:05:04.619443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607fb2e-d3f9-483b-b3a1-b0f2f58a336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:05:04.620031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607fb2e-d3f9-483b-b3a1-b0f2f58a336d None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:05:04.620303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607fb2e-d3f9-483b-b3a1-b0f2f58a336d None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:05:04.620565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607fb2e-d3f9-483b-b3a1-b0f2f58a336d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:05:04.621266 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607fb2e-d3f9-483b-b3a1-b0f2f58a336d None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=a4d007544bbc44e3935ffee978cb670f, user.id=75b229be4de4495e91ff73db474c9c90) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:05:04.621563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607fb2e-d3f9-483b-b3a1-b0f2f58a336d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:05:04.622985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607fb2e-d3f9-483b-b3a1-b0f2f58a336d None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:05:04.628442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 608/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:05:04 2022] GET /identity/v3/role_assignments?user.id=75b229be4de4495e91ff73db474c9c90&scope.project.id=a4d007544bbc44e3935ffee978cb670f&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:05:14.468851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=62 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:05:14.625911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=59 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:18.697255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fa2b9-d6af-4dbd-ae4e-6e6087dd9b0a None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:18.697822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fa2b9-d6af-4dbd-ae4e-6e6087dd9b0a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:18.697822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fa2b9-d6af-4dbd-ae4e-6e6087dd9b0a None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:18.699712 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 593/1201] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:18.709360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344254-3fa3-4c5f-b560-1ca874d3bffe None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:18.709895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344254-3fa3-4c5f-b560-1ca874d3bffe None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:18.709895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0344254-3fa3-4c5f-b560-1ca874d3bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:18.729570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c0344254-3fa3-4c5f-b560-1ca874d3bffe None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:18.730792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c0344254-3fa3-4c5f-b560-1ca874d3bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:18.795862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 609/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:19.066979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fa2b9-d6af-4dbd-ae4e-6e6087dd9b0a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:19.169387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6fa2b9-d6af-4dbd-ae4e-6e6087dd9b0a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:19.203782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:19.204392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:19.204665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:19.204936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:19.215493 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:19.219394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:19.231250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:19.232461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:19.233655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:19.233655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:19.233655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:19.233655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 594/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:19.238616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0344254-3fa3-4c5f-b560-1ca874d3bffe None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:19.339974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0344254-3fa3-4c5f-b560-1ca874d3bffe None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:19.368454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:19.375024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48641f7c-b468-496e-89cb-32fae0985cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:19.375625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48641f7c-b468-496e-89cb-32fae0985cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:19.375917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48641f7c-b468-496e-89cb-32fae0985cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:19.376168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48641f7c-b468-496e-89cb-32fae0985cb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:19.376797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48641f7c-b468-496e-89cb-32fae0985cb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:19.377095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48641f7c-b468-496e-89cb-32fae0985cb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:19.378468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48641f7c-b468-496e-89cb-32fae0985cb9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:19.389464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 610/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:19.405342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:19.450265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:19.495831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242d39f-5b36-4dd5-b252-be39974d904a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:19.528806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9d024-bcb7-429c-8c38-d93dc6ccd65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:19.529391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9d024-bcb7-429c-8c38-d93dc6ccd65b None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:19.529666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9d024-bcb7-429c-8c38-d93dc6ccd65b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:19.529931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9d024-bcb7-429c-8c38-d93dc6ccd65b None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:19.530585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9d024-bcb7-429c-8c38-d93dc6ccd65b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:19.531988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9d024-bcb7-429c-8c38-d93dc6ccd65b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:19.638582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 595/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Fri Sep 9 23:06:19 2022] POST /identity/v3/users => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:06:21.864428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6ae8-91e1-436c-8b4a-08359608381f None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:21.864994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6ae8-91e1-436c-8b4a-08359608381f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:21.864994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa6ae8-91e1-436c-8b4a-08359608381f None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:21.866297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 611/1206] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:21.871553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db190d6-cab9-4396-b78d-c9c5ba67a37b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:21.872197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db190d6-cab9-4396-b78d-c9c5ba67a37b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:21.872197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db190d6-cab9-4396-b78d-c9c5ba67a37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:21.892056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0db190d6-cab9-4396-b78d-c9c5ba67a37b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:21.893228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0db190d6-cab9-4396-b78d-c9c5ba67a37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:21.958530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 596/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:22.234487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa6ae8-91e1-436c-8b4a-08359608381f None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:22.336580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa6ae8-91e1-436c-8b4a-08359608381f None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:22.370049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4043e20-446b-46dd-9687-787fc1f1450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:22.370683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4043e20-446b-46dd-9687-787fc1f1450a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:22.370959 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4043e20-446b-46dd-9687-787fc1f1450a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:22.371228 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4043e20-446b-46dd-9687-787fc1f1450a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:22.381355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4043e20-446b-46dd-9687-787fc1f1450a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-e4043e20-446b-46dd-9687-787fc1f1450a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:06:22.386053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:22.387844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:22.388741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 612/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:06:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:22.394014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db190d6-cab9-4396-b78d-c9c5ba67a37b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:22.435408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:22.493844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db190d6-cab9-4396-b78d-c9c5ba67a37b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:22.527039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:22.527626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:22.528472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:22.528737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:22.529381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:22.529673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:22.531157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:22.538884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 597/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:06:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:22.545480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4043e20-446b-46dd-9687-787fc1f1450a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:22.596413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:22.644314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4043e20-446b-46dd-9687-787fc1f1450a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:22.677111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:22.677718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:22.677990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:22.678254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:22.689269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:22.690708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:22.705183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:22.706323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:22.707517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:22.707517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:22.707517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:22.707517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 613/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:22.712519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:22.808100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:22.811871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569dbff-473c-484b-93e6-49443f6e4645 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:22.845807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:22.846424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:22.846700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:22.846966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:22.847639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:22.847933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:22.849283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:22.860845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 598/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:22.866826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:22.966298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f87fc1-4185-4570-a8b8-80d044433486 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:22.967851 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:22.999787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.000400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.000673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.000944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.007635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.009037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:06:23.020749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:23.021878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:23.023262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 614/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:23.027702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.129563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399714d5-2a11-40b9-ba1c-985ea7da380a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.162460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.163055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.163328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.163593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.164300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:23.164598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.166022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.183104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 599/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:23.189583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.286217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03960c-5c4c-4fdd-8ff2-97f1dea27e67 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.321510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13618552-c080-4b15-9422-bfed0f71f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.324915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13618552-c080-4b15-9422-bfed0f71f83a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.324915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13618552-c080-4b15-9422-bfed0f71f83a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.324915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13618552-c080-4b15-9422-bfed0f71f83a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.352687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:23.389858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13618552-c080-4b15-9422-bfed0f71f83a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.391280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13618552-c080-4b15-9422-bfed0f71f83a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.487229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 615/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:06:23.492794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.511632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:23.591647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67362c-ba75-4d74-aadc-392d1c25cc25 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.625010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.625632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.625907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.626170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.636075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.637654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:23.651975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:23.653807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:23.655132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:23.655132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:23.655132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:23.655132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 600/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:23.659412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13618552-c080-4b15-9422-bfed0f71f83a None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.720741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:23.758064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13618552-c080-4b15-9422-bfed0f71f83a None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.792424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.793039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.793333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.794775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.795467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:23.795965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.797840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.810002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 616/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:23.815859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:23.884853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:23.915213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c49874-fb88-4cce-b1f9-3842cd9f608a None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:23.948488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:23.949096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:23.949369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:23.949799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:23.956176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:23.958503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:06:23.969978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:23.970984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:23.971931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 601/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:23.977259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:24.076604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8e4bb-c911-4844-93b7-ea2ab73dcfec None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:24.098746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:24.110285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18adac2-6650-4cda-ae76-90aa16e7615a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:24.110903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18adac2-6650-4cda-ae76-90aa16e7615a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:24.111172 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18adac2-6650-4cda-ae76-90aa16e7615a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:24.111435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18adac2-6650-4cda-ae76-90aa16e7615a None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:24.112105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18adac2-6650-4cda-ae76-90aa16e7615a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:24.112400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18adac2-6650-4cda-ae76-90aa16e7615a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:24.113792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18adac2-6650-4cda-ae76-90aa16e7615a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:24.126427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 617/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:24.132819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:24.233072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d280f-ddcd-4cf8-8c9d-854d62e0875f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:24.262600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:24.269169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9735d-710b-41ab-aa55-ff419f05cdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:24.270108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9735d-710b-41ab-aa55-ff419f05cdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:24.270501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9735d-710b-41ab-aa55-ff419f05cdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:24.270816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9735d-710b-41ab-aa55-ff419f05cdd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:24.271633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9735d-710b-41ab-aa55-ff419f05cdd3 None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=a418a2d11a234b4586cb57f1c9a4d780) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:24.272012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9735d-710b-41ab-aa55-ff419f05cdd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:24.273713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9735d-710b-41ab-aa55-ff419f05cdd3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:24.281075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 602/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:06:24 2022] GET /identity/v3/role_assignments?user.id=a418a2d11a234b4586cb57f1c9a4d780&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:26.421160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43898a9c-e22f-497f-b4f2-6a4c7e3fe2e2 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:26.421628 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43898a9c-e22f-497f-b4f2-6a4c7e3fe2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:26.421742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43898a9c-e22f-497f-b4f2-6a4c7e3fe2e2 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:26.423048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 618/1220] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:26.428244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876a21-6697-4c6b-90ee-bdc090e1ff79 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:26.428673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876a21-6697-4c6b-90ee-bdc090e1ff79 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:26.428906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c876a21-6697-4c6b-90ee-bdc090e1ff79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:26.446719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6c876a21-6697-4c6b-90ee-bdc090e1ff79 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:26.447884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6c876a21-6697-4c6b-90ee-bdc090e1ff79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:26.514749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 603/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:26.785439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43898a9c-e22f-497f-b4f2-6a4c7e3fe2e2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:26.893621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43898a9c-e22f-497f-b4f2-6a4c7e3fe2e2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:26.927227 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f129-d53d-474a-bb94-2577e1d67826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:26.927882 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f129-d53d-474a-bb94-2577e1d67826 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:26.928153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f129-d53d-474a-bb94-2577e1d67826 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:26.928422 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9f129-d53d-474a-bb94-2577e1d67826 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:26.939120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c9f129-d53d-474a-bb94-2577e1d67826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-e2c9f129-d53d-474a-bb94-2577e1d67826 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:06:26.943897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:26.945097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:26.946173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 619/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:06:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:26.951265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876a21-6697-4c6b-90ee-bdc090e1ff79 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:27.049180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c876a21-6697-4c6b-90ee-bdc090e1ff79 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:27.072294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:27.083395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:27.083997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:27.084279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:27.084568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:27.085204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:27.085503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:27.087154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:27.095521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 604/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:27.101329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f129-d53d-474a-bb94-2577e1d67826 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:27.144067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:27.197866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9f129-d53d-474a-bb94-2577e1d67826 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:27.232043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:27.232668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:27.232945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:27.233217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:27.243207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:27.244648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:27.260356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:27.261445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:27.263395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:27.263395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:27.263395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:27.263395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 620/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:27.267923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:27.374614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04e2a7d-e2e4-4a76-9182-626bdf7e02ea None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:27.408982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:27.409567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:27.409849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:27.410117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:27.410774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:27.411071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:27.412431 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:27.424753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 605/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:27.430564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:27.525475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:27.532225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca0f41-dc39-49f0-9bf2-8fd7ed18cc69 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:27.565840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:27.566608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:27.566889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:27.567241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:27.574171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:27.576324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:06:27.588442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:27.589625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:27.590756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 621/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:27.595856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:27.643800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:27.696669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc6bf1-3908-4515-ace5-913abeb70b8b None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:27.732585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:27.733211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:27.733457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:27.733815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:27.734481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:27.734785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:27.736195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:27.752996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 606/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Sep 9 23:06:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:27.758956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:27.858932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de6341-fbd1-4835-88ed-4288a91ae462 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:27.892268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:27.892871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:27.893144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:27.893408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:27.903718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:27.905105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:27.918677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:27.919800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:27.920890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:27.920890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:27.920890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:27.920890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 622/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:27.925646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:28.031682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ecb41-4de6-4b9a-af16-e30ea30eeea3 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:28.034128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:28.065734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:28.066331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:28.066631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:28.066894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:28.067517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:28.067810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:28.069146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:28.080631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 607/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:28.086594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:28.095889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:28.183885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29c1a0-2aa4-425f-8f17-34edc50e9190 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:28.217030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:28.217642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:28.217915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:28.218234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:28.224638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:28.226616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:06:28.238650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:28.239803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:28.240933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 623/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:28.245648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:28.345525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8eda8d-e31b-4f8a-aa73-24bdcf8b52db None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:28.378681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22827bc3-07c9-4f03-b01e-f97aa34897d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:28.379272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827bc3-07c9-4f03-b01e-f97aa34897d2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:28.379543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827bc3-07c9-4f03-b01e-f97aa34897d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:28.379813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22827bc3-07c9-4f03-b01e-f97aa34897d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:28.380480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827bc3-07c9-4f03-b01e-f97aa34897d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:28.380773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827bc3-07c9-4f03-b01e-f97aa34897d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:28.382184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22827bc3-07c9-4f03-b01e-f97aa34897d2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:28.394542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 608/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:28.400585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:28.468549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:28.502621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2215daf-182d-47ab-bbda-deccaf636e25 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:28.533833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a10cd1-bc70-4cfe-ad4b-3a06ee40900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:28.534502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10cd1-bc70-4cfe-ad4b-3a06ee40900b None admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:28.534776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10cd1-bc70-4cfe-ad4b-3a06ee40900b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:28.535042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a10cd1-bc70-4cfe-ad4b-3a06ee40900b None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/a418a2d11a234b4586cb57f1c9a4d780/roles/8dc6b765036246cfaf05b6553832c86a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:28.549099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a10cd1-bc70-4cfe-ad4b-3a06ee40900b None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=a418a2d11a234b4586cb57f1c9a4d780, role_id=8dc6b765036246cfaf05b6553832c86a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:28.553704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a10cd1-bc70-4cfe-ad4b-3a06ee40900b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:28.645962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 624/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:06:28 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/a418a2d11a234b4586cb57f1c9a4d780/roles/8dc6b765036246cfaf05b6553832c86a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:30.826269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc8f0f-e8d8-490a-978e-2ade8af1cb21 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:30.826824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc8f0f-e8d8-490a-978e-2ade8af1cb21 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:30.826943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc8f0f-e8d8-490a-978e-2ade8af1cb21 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:30.828236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 609/1233] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:30.833700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6497-e649-491e-ab4e-740ce008c090 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:30.834108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6497-e649-491e-ab4e-740ce008c090 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:30.834240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6497-e649-491e-ab4e-740ce008c090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:30.852858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7e7b6497-e649-491e-ab4e-740ce008c090 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:30.854038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7e7b6497-e649-491e-ab4e-740ce008c090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:30.922539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 625/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:31.203436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc8f0f-e8d8-490a-978e-2ade8af1cb21 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:31.229313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:31.305814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc8f0f-e8d8-490a-978e-2ade8af1cb21 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:31.339603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8352c1-58be-482b-bd08-4faa370b6839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:31.340220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8352c1-58be-482b-bd08-4faa370b6839 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:31.340492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8352c1-58be-482b-bd08-4faa370b6839 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:31.340757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8352c1-58be-482b-bd08-4faa370b6839 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:31.351606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8352c1-58be-482b-bd08-4faa370b6839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-7e8352c1-58be-482b-bd08-4faa370b6839 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:06:31.356401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:31.357578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:31.358587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 610/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Fri Sep 9 23:06:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:31.363904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b6497-e649-491e-ab4e-740ce008c090 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:31.410397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:31.464625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b6497-e649-491e-ab4e-740ce008c090 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:31.498222 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:31.498832 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:31.499190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:31.499454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:31.500132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] RBAC: Adding query filter params (name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:31.500424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:31.501755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:31.509454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 626/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Fri Sep 9 23:06:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:31.515339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8352c1-58be-482b-bd08-4faa370b6839 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:31.597954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:31.614045 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8352c1-58be-482b-bd08-4faa370b6839 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:31.646697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:31.647313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:31.647588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:31.647856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:31.658448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:31.659882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:31.674580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:31.675759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:31.676905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:31.676905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:31.676905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:31.678506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 611/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:31.681491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:31.780937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0909dc-f048-4ed1-b396-c19206bd041b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:31.789539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:31.813558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:31.814144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:31.814452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:31.814718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:31.815338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:31.815630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:31.816996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:31.828221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 627/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:31.834036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:31.931715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c955aa-0fb9-4f83-803d-c7e202dd2898 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:31.965268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:31.965881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:31.966740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:31.967008 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:31.969727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:31.974773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:31.976214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:06:31.987718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:31.993407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:31.994422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 612/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:31.995027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:32.094265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1f28f-bc97-41be-ae22-17fb08e1304c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:32.128248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:32.128846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:32.129120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:32.129388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:32.130143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:32.130479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:32.131877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:32.150512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 628/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Sep 9 23:06:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:32.157486 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:32.256216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559afc0e-ff5f-4e52-8cf8-fec73dc1e8c0 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:32.289527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ed2c-ac5f-426b-9a29-5f6b7730439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:32.290117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ed2c-ac5f-426b-9a29-5f6b7730439f None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:32.290416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ed2c-ac5f-426b-9a29-5f6b7730439f None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:32.290686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8659ed2c-ac5f-426b-9a29-5f6b7730439f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:32.351860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:32.358227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659ed2c-ac5f-426b-9a29-5f6b7730439f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:32.359688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8659ed2c-ac5f-426b-9a29-5f6b7730439f None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:32.455850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 613/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:06:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:06:32.462023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:32.488799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:32.561549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b1b6a-1438-4152-a4c2-d557a6f8ff8b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:32.594595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:32.595182 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:32.595454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:32.595717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:32.605204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:32.606607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:32.620315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:32.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:32.622912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:32.622912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:32.622912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:32.622912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 629/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:32.627713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ed2c-ac5f-426b-9a29-5f6b7730439f None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:32.719610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:32.725498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8659ed2c-ac5f-426b-9a29-5f6b7730439f None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:32.758071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:32.758696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:32.758969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:32.759236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:32.759865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:32.760162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:32.761501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:32.772802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 614/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:32.778549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:32.855753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:32.877208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd364a-e1ca-424c-96e7-e822132fa988 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:32.912357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:32.913003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:32.913267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:32.913531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:32.920597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:32.922034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:06:32.933219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:32.934276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:32.935782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 630/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:06:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:32.940670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:33.044999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f41330-ff4a-4912-b862-36ce5dacb2b7 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:33.077955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f962f6-7922-4df1-8301-ae79990fc206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:33.078576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f962f6-7922-4df1-8301-ae79990fc206 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:33.078851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f962f6-7922-4df1-8301-ae79990fc206 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:33.079116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f962f6-7922-4df1-8301-ae79990fc206 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:33.079907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f962f6-7922-4df1-8301-ae79990fc206 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:33.080206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f962f6-7922-4df1-8301-ae79990fc206 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:33.081616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f962f6-7922-4df1-8301-ae79990fc206 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:33.094008 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 615/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:06:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:33.100375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:33.197327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf93a52-cced-4025-be1b-cf638792c948 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:33.229878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a233a48-32a5-430c-9670-910a65172778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:33.230534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233a48-32a5-430c-9670-910a65172778 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:33.230808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233a48-32a5-430c-9670-910a65172778 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:33.231076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a233a48-32a5-430c-9670-910a65172778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:33.231909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a233a48-32a5-430c-9670-910a65172778 None admin] RBAC: Adding query filter params (role.id=8dc6b765036246cfaf05b6553832c86a, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=a418a2d11a234b4586cb57f1c9a4d780) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:33.232219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a233a48-32a5-430c-9670-910a65172778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:33.233624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a233a48-32a5-430c-9670-910a65172778 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:33.239192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 631/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:06:33 2022] GET /identity/v3/role_assignments?user.id=a418a2d11a234b4586cb57f1c9a4d780&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=8dc6b765036246cfaf05b6553832c86a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:35.458786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bbd41-d187-4b13-bab6-71b819f5102b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:35.459361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bbd41-d187-4b13-bab6-71b819f5102b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:35.459361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bbd41-d187-4b13-bab6-71b819f5102b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:35.460691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 616/1247] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:35.466537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9debb7-2c5b-4b96-bdfc-11b50678109d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:35.466970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9debb7-2c5b-4b96-bdfc-11b50678109d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:35.467309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9debb7-2c5b-4b96-bdfc-11b50678109d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:35.473589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:35.486260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3a9debb7-2c5b-4b96-bdfc-11b50678109d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:35.487489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3a9debb7-2c5b-4b96-bdfc-11b50678109d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:35.554434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 632/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:35.835509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bbd41-d187-4b13-bab6-71b819f5102b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:35.854478 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:35.933959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bbd41-d187-4b13-bab6-71b819f5102b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:35.966676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e58d4b-e7ef-4e6f-90a3-0c487ec16a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:35.967283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e58d4b-e7ef-4e6f-90a3-0c487ec16a61 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:35.967556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e58d4b-e7ef-4e6f-90a3-0c487ec16a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:35.967820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e58d4b-e7ef-4e6f-90a3-0c487ec16a61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:35.977384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e58d4b-e7ef-4e6f-90a3-0c487ec16a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-08e58d4b-e7ef-4e6f-90a3-0c487ec16a61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:06:35.982720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:35.983943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:35.984975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 617/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:06:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:35.989834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9debb7-2c5b-4b96-bdfc-11b50678109d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.089600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9debb7-2c5b-4b96-bdfc-11b50678109d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.118924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:36.124515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.125104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.125379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.125647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.126292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:36.126617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:36.127963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:36.135707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 633/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:06:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:36.142105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e58d4b-e7ef-4e6f-90a3-0c487ec16a61 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.223110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:36.245560 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e58d4b-e7ef-4e6f-90a3-0c487ec16a61 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.279747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.280359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.280632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.280898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.292860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:36.295014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:36.309308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:36.315106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:36.316060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:36.316060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:36.316060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:36.316060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 618/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:36.316546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.416094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1e097-a42b-4613-91c1-46af2d2eaabe None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.449322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.449915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.450189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.450549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.451426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:36.451726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:36.453107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:36.464698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 634/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:36.472309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.569856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b44a6-3f7a-4e67-a553-f289ed3a2e7c None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.596018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:36.603577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.604188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.604480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.604761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.611774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:36.613293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:06:36.624205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:36.625256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:36.626205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 619/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:36.631229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.690391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:36.733809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f86c06-0ba7-4811-ad73-5a5621699627 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.765976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.766590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.766860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.767121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.767833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:36.768129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:36.769519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:36.786492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 635/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Sep 9 23:06:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:36.792390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:36.890947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464752b8-c641-4414-a927-ed7944aee464 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:36.926254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31deb-cdbe-4284-a888-e0a1e0d12231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:36.927014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31deb-cdbe-4284-a888-e0a1e0d12231 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:36.927288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31deb-cdbe-4284-a888-e0a1e0d12231 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:36.927559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31deb-cdbe-4284-a888-e0a1e0d12231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:36.989565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:37.002091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac31deb-cdbe-4284-a888-e0a1e0d12231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.007177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac31deb-cdbe-4284-a888-e0a1e0d12231 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.099109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 620/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:06:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:06:37.105193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.204709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc072b02-3c3b-4a67-b6a7-70d440ccfcfe None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.217913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:37.238753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.239370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.239693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.239911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.250616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.252059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:37.267028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:37.272446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:37.273510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:37.273510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:37.273510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:37.273510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 636/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:37.274103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31deb-cdbe-4284-a888-e0a1e0d12231 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.360518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:37.372952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31deb-cdbe-4284-a888-e0a1e0d12231 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.405388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.405974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.406247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.406548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.407365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:37.407660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.409003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.420920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 621/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:37.426649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.523549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e9a81-d479-4151-b5e5-735f4567643f None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.556975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.557639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.557916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.558182 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.565469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.566929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:06:37.578677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:37.584709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:37.585858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 637/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:37.586867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.683816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393dc6b-e106-45bd-88c1-756d8cf494c2 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.717527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d741f-ec7b-420e-9520-e4102d57d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.718120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d741f-ec7b-420e-9520-e4102d57d820 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.718417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d741f-ec7b-420e-9520-e4102d57d820 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.718692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d741f-ec7b-420e-9520-e4102d57d820 None admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.719363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d741f-ec7b-420e-9520-e4102d57d820 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:37.719666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d741f-ec7b-420e-9520-e4102d57d820 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.721116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d741f-ec7b-420e-9520-e4102d57d820 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.733494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 622/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:37.739865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:37.765204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:37.837066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de831c3-e865-4ce8-8099-fa88d8aaadda None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:37.871540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038bf04-28bd-4901-9166-4edacaa6de52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:37.872144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038bf04-28bd-4901-9166-4edacaa6de52 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:37.872476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038bf04-28bd-4901-9166-4edacaa6de52 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:37.872743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038bf04-28bd-4901-9166-4edacaa6de52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:37.873497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038bf04-28bd-4901-9166-4edacaa6de52 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=a418a2d11a234b4586cb57f1c9a4d780) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:37.873799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038bf04-28bd-4901-9166-4edacaa6de52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:37.875240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038bf04-28bd-4901-9166-4edacaa6de52 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:37.880471 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 638/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:06:37 2022] GET /identity/v3/role_assignments?user.id=a418a2d11a234b4586cb57f1c9a4d780&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:40.034226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641c6bf-d9f5-4c60-b27e-57e276bb4b29 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:40.034783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641c6bf-d9f5-4c60-b27e-57e276bb4b29 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:40.034783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641c6bf-d9f5-4c60-b27e-57e276bb4b29 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:40.036286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 623/1261] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:40.041511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dda502-a1f4-4eb6-b694-96da7ecfa1a8 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:40.041926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dda502-a1f4-4eb6-b694-96da7ecfa1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:40.042078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dda502-a1f4-4eb6-b694-96da7ecfa1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:40.061481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-34dda502-a1f4-4eb6-b694-96da7ecfa1a8 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:40.062709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-34dda502-a1f4-4eb6-b694-96da7ecfa1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:40.127507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 639/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:40.394535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641c6bf-d9f5-4c60-b27e-57e276bb4b29 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:40.406080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:40.497498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641c6bf-d9f5-4c60-b27e-57e276bb4b29 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:40.532571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eab78-4990-4465-b06d-2a9c5be0b0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:40.533191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eab78-4990-4465-b06d-2a9c5be0b0cd None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:40.533468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eab78-4990-4465-b06d-2a9c5be0b0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:40.533736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eab78-4990-4465-b06d-2a9c5be0b0cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:40.543597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7eab78-4990-4465-b06d-2a9c5be0b0cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-4d7eab78-4990-4465-b06d-2a9c5be0b0cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:06:40.548433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:40.549540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:40.550634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 624/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:06:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:40.555591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dda502-a1f4-4eb6-b694-96da7ecfa1a8 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:40.574262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:40.671216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dda502-a1f4-4eb6-b694-96da7ecfa1a8 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:40.711800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:40.712396 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:40.712670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:40.712937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:40.713591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:40.713888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:40.715257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:40.723650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 640/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:06:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:40.729423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eab78-4990-4465-b06d-2a9c5be0b0cd None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:40.799920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:40.832388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eab78-4990-4465-b06d-2a9c5be0b0cd None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:40.866019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:40.866675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:40.866932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:40.867196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:40.877507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:40.879038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:40.892743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:40.893867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:40.895109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:40.895109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:40.895109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:40.895109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 625/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:40.899652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:40.987785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:41.007649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8ef487-98cf-40c4-820e-92122b3000fc None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:41.041997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:41.042638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:41.042876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:41.043141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:41.043762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:41.044054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:41.045390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:41.056397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 641/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:41.062511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:41.161901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f5f67-50b5-41ba-9614-5ddba34978b6 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:41.180810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:41.194303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:41.194950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:41.195227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:41.195498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:41.203289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:41.204714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:06:41.215757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:41.217378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:41.225794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 626/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:41.226575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:41.324504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f31900-1c7b-4dbb-8f64-57b890273305 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:41.358315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:41.358923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:41.359199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:41.359468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:41.360173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:41.360472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:41.361877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:41.370678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:41.379146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 642/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Sep 9 23:06:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:41.384978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:41.488922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a5fe-28d8-47e4-af08-9fac5955fe26 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:41.521246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:41.521898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:41.522181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:41.522498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:41.533205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:41.534672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:41.549174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:41.550761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:41.574912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:41.574912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:41.574912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:41.574912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 627/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:06:41.575209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:41.654356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fd199-5821-4f04-8681-70ba78607625 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:41.688472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:41.689066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:41.689339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:41.689605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:41.690380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:41.690702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:41.692061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:41.704076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 643/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:41.709764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:41.741678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:41.808638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b02f7-0914-4cdb-911e-0d8b7501193e None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:41.843989 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:41.844546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:41.844819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:41.845085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:41.851812 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:41.853520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:06:41.865264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:41.869451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:41.870748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 628/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:06:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:41.873041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:41.929891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:41.969738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac2821-0e71-4d14-80b7-e37df9b2db5c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:42.003586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1536cb-f1ac-4d14-9fdc-66c05a30d687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:42.004204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1536cb-f1ac-4d14-9fdc-66c05a30d687 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:42.004577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1536cb-f1ac-4d14-9fdc-66c05a30d687 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:42.004850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1536cb-f1ac-4d14-9fdc-66c05a30d687 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:42.005528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1536cb-f1ac-4d14-9fdc-66c05a30d687 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:42.005827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1536cb-f1ac-4d14-9fdc-66c05a30d687 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:42.007252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1536cb-f1ac-4d14-9fdc-66c05a30d687 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:42.019101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 644/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:42.026507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:42.121060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:42.133069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494a502-b43f-4f7c-8315-640a3e1ccdf8 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:42.167071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e97e5-5b22-4e1d-8bcb-7fd37c555465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:42.167849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e97e5-5b22-4e1d-8bcb-7fd37c555465 None admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:42.168126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e97e5-5b22-4e1d-8bcb-7fd37c555465 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:42.168395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e97e5-5b22-4e1d-8bcb-7fd37c555465 None admin] PATH_INFO: `/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/a418a2d11a234b4586cb57f1c9a4d780/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:42.170934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e97e5-5b22-4e1d-8bcb-7fd37c555465 None admin] RBAC: Authorizing `identity:create_grant(project_id=73855506219b40fdb5a097bdc5cbed81, user_id=a418a2d11a234b4586cb57f1c9a4d780, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:42.172353 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e97e5-5b22-4e1d-8bcb-7fd37c555465 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:42.268811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 629/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Fri Sep 9 23:06:42 2022] PUT /identity/v3/projects/73855506219b40fdb5a097bdc5cbed81/users/a418a2d11a234b4586cb57f1c9a4d780/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:06:44.422183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af562c-53ef-4e0e-a6ed-24b218e57f46 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:44.422632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af562c-53ef-4e0e-a6ed-24b218e57f46 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:44.422768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af562c-53ef-4e0e-a6ed-24b218e57f46 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:44.424105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 645/1274] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:44.429528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8bb09-f82a-4be2-9441-bd04ad5809e7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:44.429952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8bb09-f82a-4be2-9441-bd04ad5809e7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:44.430075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8bb09-f82a-4be2-9441-bd04ad5809e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:44.447847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2ff8bb09-f82a-4be2-9441-bd04ad5809e7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:44.449035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2ff8bb09-f82a-4be2-9441-bd04ad5809e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:44.514629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 630/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:44.784674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af562c-53ef-4e0e-a6ed-24b218e57f46 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:44.885678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af562c-53ef-4e0e-a6ed-24b218e57f46 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:44.902934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:44.920152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b194c0-05c6-47f5-a3f8-ec27ab088f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:44.920767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b194c0-05c6-47f5-a3f8-ec27ab088f26 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:44.921044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b194c0-05c6-47f5-a3f8-ec27ab088f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:44.921309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b194c0-05c6-47f5-a3f8-ec27ab088f26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:44.931447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b194c0-05c6-47f5-a3f8-ec27ab088f26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-96b194c0-05c6-47f5-a3f8-ec27ab088f26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 09 23:06:44.936205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:44.937389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:44.938328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 646/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Fri Sep 9 23:06:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:44.943484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8bb09-f82a-4be2-9441-bd04ad5809e7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:45.025719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:45.044352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8bb09-f82a-4be2-9441-bd04ad5809e7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:45.077608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:45.078202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:45.078511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:45.078781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:45.079426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:45.079729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:45.081080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:45.088998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 631/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Fri Sep 9 23:06:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:45.095536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b194c0-05c6-47f5-a3f8-ec27ab088f26 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:45.192509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b194c0-05c6-47f5-a3f8-ec27ab088f26 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:45.225998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:45.226639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:45.226910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:45.227176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:45.237276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:45.238741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:45.253901 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:45.255081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:45.256271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:45.256271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:45.256271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:45.256271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 647/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:45.260887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:45.360838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e9048a-fdd1-46f1-891a-47f0c1b1ac94 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:45.393448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:45.394040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:45.394312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:45.394608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:45.395244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:45.395538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:45.396886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:45.409009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 632/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:45.421816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:45.451596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:45.519915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75d88-acbe-4750-bb70-9d5d60560547 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:45.552687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:45.553306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:45.553574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:45.553841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] PATH_INFO: `/v3/users/placement` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:45.560613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:45.566329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 09 23:06:45.573502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:45.574676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:45.576812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 648/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:45.581026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:45.584967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:45.680606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bdf00b-f226-4fa6-b325-1577dc808076 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:45.713772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:45.714391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:45.714693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:45.714960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:45.715680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:45.716601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:45.718579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:45.735729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 633/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Fri Sep 9 23:06:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:45.742420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:45.837263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:45.847506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4b564-dc0c-4b32-a282-0f0e4473f2df None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:45.882456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c6dae-bde4-43e6-bd72-5cf2d268a42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:45.883082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c6dae-bde4-43e6-bd72-5cf2d268a42c None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:45.883354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c6dae-bde4-43e6-bd72-5cf2d268a42c None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:45.883619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c6dae-bde4-43e6-bd72-5cf2d268a42c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:45.960104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c6dae-bde4-43e6-bd72-5cf2d268a42c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:45.962116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c6dae-bde4-43e6-bd72-5cf2d268a42c None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:46.058593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:46.068931 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 649/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Fri Sep 9 23:06:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 09 23:06:46.075254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:46.141678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:46.183669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b54d72-54c1-44fa-a523-bac092f0dacc None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:46.223719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:46.224275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:46.224553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:46.224850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:46.236317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:46.237758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 09 23:06:46.258582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:46.260385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 09 23:06:46.261532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 09 23:06:46.261532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 09 23:06:46.261532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:46.261532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 634/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:46.268489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c6dae-bde4-43e6-bd72-5cf2d268a42c None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:46.370708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c6dae-bde4-43e6-bd72-5cf2d268a42c None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:46.411963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:46.412579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:46.412850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:46.413110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:46.413899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:46.414187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:46.415853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:46.428009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 650/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Fri Sep 9 23:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:46.433662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:46.532956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c715f5-e126-4f91-be62-7eef94cd9440 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:46.546347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:46.565922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:46.566584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:46.566858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:46.567125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:46.573437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:46.574880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 09 23:06:46.587720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:46.590161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: ERROR keystone.server.flask.application Sep 09 23:06:46.591892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 635/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Fri Sep 9 23:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:46.594870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:46.644456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:46.694034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379ec8-61f5-4f26-991d-c749c39f9c7b None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:46.727160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38365b4-42cb-483c-aa07-a0ddbbf73703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:46.727753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38365b4-42cb-483c-aa07-a0ddbbf73703 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:46.728029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38365b4-42cb-483c-aa07-a0ddbbf73703 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:46.728297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38365b4-42cb-483c-aa07-a0ddbbf73703 None admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:46.728973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38365b4-42cb-483c-aa07-a0ddbbf73703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:46.729270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38365b4-42cb-483c-aa07-a0ddbbf73703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:46.730706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38365b4-42cb-483c-aa07-a0ddbbf73703 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:46.742843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 651/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Fri Sep 9 23:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:46.752574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:46.846111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12222e4-62a2-4533-a5f1-808bb14a3732 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:46.878953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cb0ab-58d0-46d9-829e-6c13f270e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:46.884480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb0ab-58d0-46d9-829e-6c13f270e037 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:46.884480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb0ab-58d0-46d9-829e-6c13f270e037 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:46.884480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cb0ab-58d0-46d9-829e-6c13f270e037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:46.884480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cb0ab-58d0-46d9-829e-6c13f270e037 None admin] RBAC: Adding query filter params (role.id=c935d5f3d40f451d9e734a3fd8077e22, scope.project.id=73855506219b40fdb5a097bdc5cbed81, user.id=a418a2d11a234b4586cb57f1c9a4d780) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:46.884480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cb0ab-58d0-46d9-829e-6c13f270e037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:46.884480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3cb0ab-58d0-46d9-829e-6c13f270e037 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:46.888155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 636/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Fri Sep 9 23:06:46 2022] GET /identity/v3/role_assignments?user.id=a418a2d11a234b4586cb57f1c9a4d780&scope.project.id=73855506219b40fdb5a097bdc5cbed81&role.id=c935d5f3d40f451d9e734a3fd8077e22 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:49.037439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88c44d-0275-4ea0-bcdc-c16abfe170fb None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:49.037995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88c44d-0275-4ea0-bcdc-c16abfe170fb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:49.037995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88c44d-0275-4ea0-bcdc-c16abfe170fb None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:49.039321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 652/1288] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:06:49.044980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564e20c-8fbb-48c5-8e68-bc85817730df None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:49.045420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564e20c-8fbb-48c5-8e68-bc85817730df None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:49.045554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564e20c-8fbb-48c5-8e68-bc85817730df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:49.065311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0564e20c-8fbb-48c5-8e68-bc85817730df None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:49.066537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0564e20c-8fbb-48c5-8e68-bc85817730df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:49.097203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:49.133640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 637/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:49.411221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88c44d-0275-4ea0-bcdc-c16abfe170fb None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:49.519128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88c44d-0275-4ea0-bcdc-c16abfe170fb None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:49.529618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:49.551865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:49.552473 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:49.552747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:49.553011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:49.553574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:49.554942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:49.563937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:49.565164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:49.566010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 653/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:06:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:49.571278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564e20c-8fbb-48c5-8e68-bc85817730df None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:49.669045 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0564e20c-8fbb-48c5-8e68-bc85817730df None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:49.702795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adaa79-98e0-4c55-9841-83a4232cee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:49.703370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adaa79-98e0-4c55-9841-83a4232cee28 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:49.703647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adaa79-98e0-4c55-9841-83a4232cee28 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:49.703916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adaa79-98e0-4c55-9841-83a4232cee28 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:49.704563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adaa79-98e0-4c55-9841-83a4232cee28 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:49.704908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adaa79-98e0-4c55-9841-83a4232cee28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:49.706264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adaa79-98e0-4c55-9841-83a4232cee28 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:49.710967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 638/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Sep 9 23:06:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:49.717677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:49.821176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faae52e-bc9c-48a9-aad5-3e09578174d4 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:49.857484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309e72c-6eed-468d-b2fc-82a1fed00b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:49.858082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309e72c-6eed-468d-b2fc-82a1fed00b6a None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:49.858376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309e72c-6eed-468d-b2fc-82a1fed00b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:49.858647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309e72c-6eed-468d-b2fc-82a1fed00b6a None admin] PATH_INFO: `/v3/services` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:49.859259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309e72c-6eed-468d-b2fc-82a1fed00b6a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:49.859554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309e72c-6eed-468d-b2fc-82a1fed00b6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:49.860889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309e72c-6eed-468d-b2fc-82a1fed00b6a None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:49.865232 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 654/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Sep 9 23:06:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:06:52.026861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995787-5a8c-491a-a312-d0afad734536 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:52.027392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995787-5a8c-491a-a312-d0afad734536 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:52.027392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995787-5a8c-491a-a312-d0afad734536 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:52.029256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 639/1293] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:52.034593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a690-038b-42dd-a0eb-fb97888bbbfd None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:52.034994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a690-038b-42dd-a0eb-fb97888bbbfd None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:52.035135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7973a690-038b-42dd-a0eb-fb97888bbbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:52.053625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7973a690-038b-42dd-a0eb-fb97888bbbfd None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:52.054830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7973a690-038b-42dd-a0eb-fb97888bbbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:52.083887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:52.121313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 655/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:52.395505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71995787-5a8c-491a-a312-d0afad734536 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:52.427358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:52.496440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71995787-5a8c-491a-a312-d0afad734536 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:52.529518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5fc2f-a229-4dc1-9420-0f6f2a0ae3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:52.530126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5fc2f-a229-4dc1-9420-0f6f2a0ae3db None admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:52.530419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5fc2f-a229-4dc1-9420-0f6f2a0ae3db None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:52.530719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5fc2f-a229-4dc1-9420-0f6f2a0ae3db None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:52.531365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5fc2f-a229-4dc1-9420-0f6f2a0ae3db None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:52.532719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5fc2f-a229-4dc1-9420-0f6f2a0ae3db None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:52.609345 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 640/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Fri Sep 9 23:06:52 2022] POST /identity/v3/services => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:06:54.808822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da899b2c-162c-4e43-a969-0577401c0c23 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:54.809340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da899b2c-162c-4e43-a969-0577401c0c23 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:54.809340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da899b2c-162c-4e43-a969-0577401c0c23 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:54.811274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 656/1296] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:06:54.816601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909decf8-95f1-4900-a924-fdafa4100571 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:54.817001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909decf8-95f1-4900-a924-fdafa4100571 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:54.817150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909decf8-95f1-4900-a924-fdafa4100571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:54.835123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-909decf8-95f1-4900-a924-fdafa4100571 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:54.836348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-909decf8-95f1-4900-a924-fdafa4100571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:54.908415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 641/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:55.189072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da899b2c-162c-4e43-a969-0577401c0c23 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:55.288440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da899b2c-162c-4e43-a969-0577401c0c23 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:55.322592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:55.323210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:55.323487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:55.323756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:55.324327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:55.325732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:55.334331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:55.337970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:55.339063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 657/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:06:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 09 23:06:55.342017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909decf8-95f1-4900-a924-fdafa4100571 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:55.360273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:55.438788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909decf8-95f1-4900-a924-fdafa4100571 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:55.472225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:55.472768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:55.473067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:55.473333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:55.473974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:55.474273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:55.475657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:55.480613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 642/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Sep 9 23:06:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:55.487079 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:55.519816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:55.587192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b38fc0-8f46-4846-a555-380ad7a64da4 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:55.620544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b0140-5808-4c68-a6dc-60e389c804d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:55.621236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0140-5808-4c68-a6dc-60e389c804d9 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:55.621506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0140-5808-4c68-a6dc-60e389c804d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:55.621770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b0140-5808-4c68-a6dc-60e389c804d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:55.622480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0140-5808-4c68-a6dc-60e389c804d9 None admin] RBAC: Adding query filter params (interface=public, service_id=d8529b7a2a824b79bc405868de6e0086, region_id=RegionOne) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:55.622781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0140-5808-4c68-a6dc-60e389c804d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:55.624145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b0140-5808-4c68-a6dc-60e389c804d9 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:55.628956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 658/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Fri Sep 9 23:06:55 2022] GET /identity/v3/endpoints?service_id=d8529b7a2a824b79bc405868de6e0086&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:55.634849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:55.715286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:55.734034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bbfe4-6c54-4d66-a075-463d1e408285 None admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:55.767240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d37a58-e5a1-4c0b-9877-e32e290a050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:55.767828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d37a58-e5a1-4c0b-9877-e32e290a050b None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:55.768102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d37a58-e5a1-4c0b-9877-e32e290a050b None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:55.768371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d37a58-e5a1-4c0b-9877-e32e290a050b None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:55.768987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d37a58-e5a1-4c0b-9877-e32e290a050b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:55.770366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d37a58-e5a1-4c0b-9877-e32e290a050b None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:55.777041 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 643/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Fri Sep 9 23:06:55 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:06:57.919015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a21821-923b-46de-8f8c-e54e9551c046 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:57.919574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a21821-923b-46de-8f8c-e54e9551c046 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:57.919574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a21821-923b-46de-8f8c-e54e9551c046 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:57.920890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 659/1302] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:06:57.926154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74859734-0fb3-4be5-9bd6-f74603f2d0e1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:57.926616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74859734-0fb3-4be5-9bd6-f74603f2d0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:57.926743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74859734-0fb3-4be5-9bd6-f74603f2d0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:57.945092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-74859734-0fb3-4be5-9bd6-f74603f2d0e1 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:06:57.946260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-74859734-0fb3-4be5-9bd6-f74603f2d0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:06:58.013050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 644/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:06:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 09 23:06:58.292599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a21821-923b-46de-8f8c-e54e9551c046 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:58.387142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:58.394397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a21821-923b-46de-8f8c-e54e9551c046 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:58.428118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:58.428739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:58.430688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:58.430688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:58.430688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:58.431200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 09 23:06:58.440009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 09 23:06:58.441147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.get_or_create( Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application with Lock( Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self._enter() Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application return self.creator() Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application created_value = creator( Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: ERROR keystone.server.flask.application Sep 09 23:06:58.442515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 660/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Fri Sep 9 23:06:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 09 23:06:58.447094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74859734-0fb3-4be5-9bd6-f74603f2d0e1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:58.520877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:06:58.542963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74859734-0fb3-4be5-9bd6-f74603f2d0e1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:58.574762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7838ac-8cc7-46de-aff2-65d355bea6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:58.575352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7838ac-8cc7-46de-aff2-65d355bea6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:58.575626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7838ac-8cc7-46de-aff2-65d355bea6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:58.575892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7838ac-8cc7-46de-aff2-65d355bea6a5 None admin] PATH_INFO: `/v3/services` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:58.576497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7838ac-8cc7-46de-aff2-65d355bea6a5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:06:58.576788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7838ac-8cc7-46de-aff2-65d355bea6a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:58.578722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7838ac-8cc7-46de-aff2-65d355bea6a5 None admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:58.584162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 645/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Fri Sep 9 23:06:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:06:58.590253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:06:58.689070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e8d92-0992-4d0d-88cb-e9ca3638dca7 None admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:06:58.722234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b230d-b84a-421f-b432-b72454cbcfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:06:58.722889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b230d-b84a-421f-b432-b72454cbcfa6 None admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:06:58.723173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b230d-b84a-421f-b432-b72454cbcfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:06:58.723437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b230d-b84a-421f-b432-b72454cbcfa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:06:58.724082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b230d-b84a-421f-b432-b72454cbcfa6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:06:58.725409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b230d-b84a-421f-b432-b72454cbcfa6 None admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:06:58.846606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 661/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:06:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:07:08.581365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=30 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:07:08.841200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:50.574820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361e2fe-3653-4b22-9907-08e18d8ae374 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:50.574820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361e2fe-3653-4b22-9907-08e18d8ae374 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:50.574820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361e2fe-3653-4b22-9907-08e18d8ae374 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:50.579829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cf0f0-dbd7-4146-a0d5-a773d9f07114 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:50.580233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cf0f0-dbd7-4146-a0d5-a773d9f07114 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:50.580434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cf0f0-dbd7-4146-a0d5-a773d9f07114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:50.585216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 646/1307] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:12:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:12:50.705613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-b93cf0f0-dbd7-4146-a0d5-a773d9f07114 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:12:50.707612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b93cf0f0-dbd7-4146-a0d5-a773d9f07114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:50.841530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 662/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:12:51.135421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c982c-627b-41c7-a5bd-1734c4180fab None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:51.135883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c982c-627b-41c7-a5bd-1734c4180fab None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:51.145761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c982c-627b-41c7-a5bd-1734c4180fab None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:51.145761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 647/1309] 10.222.0.20 () {54 vars in 965 bytes} [Fri Sep 9 23:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:12:51.145979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478037b0-feda-463f-838c-a7edb28b5991 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:51.145979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478037b0-feda-463f-838c-a7edb28b5991 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:51.145979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478037b0-feda-463f-838c-a7edb28b5991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:51.240835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-478037b0-feda-463f-838c-a7edb28b5991 None None] MFA Rules not processed for user `a9da2d7ef80f4622b58c9041b65df992`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:12:51.242110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-478037b0-feda-463f-838c-a7edb28b5991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:51.271085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:51.335912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 663/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Fri Sep 9 23:12:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:12:51.345075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c982c-627b-41c7-a5bd-1734c4180fab None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:51.347193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-992c982c-627b-41c7-a5bd-1734c4180fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:51.464913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c982c-627b-41c7-a5bd-1734c4180fab None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:51.543360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991ad6c-c1ac-4fe2-a7a6-8829b1b58253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:51.543985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991ad6c-c1ac-4fe2-a7a6-8829b1b58253 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:51.544338 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991ad6c-c1ac-4fe2-a7a6-8829b1b58253 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:51.544682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991ad6c-c1ac-4fe2-a7a6-8829b1b58253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:51.546375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4991ad6c-c1ac-4fe2-a7a6-8829b1b58253 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:51.617272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:51.696921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991ad6c-c1ac-4fe2-a7a6-8829b1b58253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:51.699080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991ad6c-c1ac-4fe2-a7a6-8829b1b58253 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:51.721082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 648/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:12:54.252217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63114f4-2e06-4996-8a38-ae0d823fcb5b None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:54.252774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63114f4-2e06-4996-8a38-ae0d823fcb5b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:54.252774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63114f4-2e06-4996-8a38-ae0d823fcb5b None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:54.253999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 664/1312] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:12:54.259304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6a71a-cbf1-4cfe-a9ff-2adbd1afc61b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:54.260258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6a71a-cbf1-4cfe-a9ff-2adbd1afc61b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:54.260413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6a71a-cbf1-4cfe-a9ff-2adbd1afc61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:54.330516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-57a6a71a-cbf1-4cfe-a9ff-2adbd1afc61b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:12:54.331732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-57a6a71a-cbf1-4cfe-a9ff-2adbd1afc61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:54.433907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 649/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:12:54.724686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63114f4-2e06-4996-8a38-ae0d823fcb5b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:54.726303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a63114f4-2e06-4996-8a38-ae0d823fcb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:54.842010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63114f4-2e06-4996-8a38-ae0d823fcb5b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:54.923314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:54.925672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f802703-7f17-4fa8-b2b4-4d9a1fc630eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:54.926274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f802703-7f17-4fa8-b2b4-4d9a1fc630eb service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:54.926586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f802703-7f17-4fa8-b2b4-4d9a1fc630eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:54.926856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f802703-7f17-4fa8-b2b4-4d9a1fc630eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:54.928637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9f802703-7f17-4fa8-b2b4-4d9a1fc630eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:55.041589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f802703-7f17-4fa8-b2b4-4d9a1fc630eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:55.042956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f802703-7f17-4fa8-b2b4-4d9a1fc630eb service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:55.061446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 665/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:12:57.725617 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bb2b4-e5c2-45ac-bd34-165d011cd736 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:57.726138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bb2b4-e5c2-45ac-bd34-165d011cd736 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:57.726138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bb2b4-e5c2-45ac-bd34-165d011cd736 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:57.727439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 650/1315] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:12:57.732832 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf23752-f46a-4346-aed2-96486503528c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:57.733250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf23752-f46a-4346-aed2-96486503528c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:57.733374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf23752-f46a-4346-aed2-96486503528c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:57.798857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-bbf23752-f46a-4346-aed2-96486503528c None None] MFA Rules not processed for user `8f335b1a4ae84353bed712f1faf9aad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:12:57.800093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bbf23752-f46a-4346-aed2-96486503528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:57.852137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:57.896346 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 666/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:12:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:12:58.184376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bb2b4-e5c2-45ac-bd34-165d011cd736 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:12:58.185912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ff3bb2b4-e5c2-45ac-bd34-165d011cd736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:58.195711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:58.287413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bb2b4-e5c2-45ac-bd34-165d011cd736 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:12:58.358576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75e626-2a8f-420e-b54f-4314a3310635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:12:58.359169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75e626-2a8f-420e-b54f-4314a3310635 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:12:58.359439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75e626-2a8f-420e-b54f-4314a3310635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:12:58.360119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75e626-2a8f-420e-b54f-4314a3310635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:12:58.361711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cd75e626-2a8f-420e-b54f-4314a3310635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:12:58.423918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:12:58.469607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75e626-2a8f-420e-b54f-4314a3310635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:12:58.471057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75e626-2a8f-420e-b54f-4314a3310635 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:12:58.488831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 651/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:12:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:01.557743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c8b0-ad50-4184-a469-a701a3a5372c None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:01.558373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c8b0-ad50-4184-a469-a701a3a5372c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:01.558485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073c8b0-ad50-4184-a469-a701a3a5372c None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:01.559791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 667/1318] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:01.564952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36220-71fa-4f8b-919c-62bcb3b47e3c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:01.565329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36220-71fa-4f8b-919c-62bcb3b47e3c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:01.565505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36220-71fa-4f8b-919c-62bcb3b47e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:01.630477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-66f36220-71fa-4f8b-919c-62bcb3b47e3c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:01.631679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-66f36220-71fa-4f8b-919c-62bcb3b47e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:01.727889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 652/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:02.023236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073c8b0-ad50-4184-a469-a701a3a5372c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:02.024804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0073c8b0-ad50-4184-a469-a701a3a5372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:02.127295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073c8b0-ad50-4184-a469-a701a3a5372c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:02.200693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d58b55-5e84-4795-b1fc-6e6f5a9173cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:02.201284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58b55-5e84-4795-b1fc-6e6f5a9173cf service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:02.201576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58b55-5e84-4795-b1fc-6e6f5a9173cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:02.201842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58b55-5e84-4795-b1fc-6e6f5a9173cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:02.203457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-39d58b55-5e84-4795-b1fc-6e6f5a9173cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:02.210119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:02.311978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d58b55-5e84-4795-b1fc-6e6f5a9173cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:02.313379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d58b55-5e84-4795-b1fc-6e6f5a9173cf service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:02.332176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 668/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:05.500194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d134-c627-4d8e-99d5-2d59957ec339 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:05.500921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d134-c627-4d8e-99d5-2d59957ec339 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:05.500921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a1d134-c627-4d8e-99d5-2d59957ec339 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:05.502181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 653/1321] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:05.507613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452c07-ba74-48be-9ec6-8e48fd33e7a9 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:05.508025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452c07-ba74-48be-9ec6-8e48fd33e7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:05.508166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452c07-ba74-48be-9ec6-8e48fd33e7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:05.582468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-55452c07-ba74-48be-9ec6-8e48fd33e7a9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:05.583731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-55452c07-ba74-48be-9ec6-8e48fd33e7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:05.646866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:05.690336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 669/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:05.981096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1d134-c627-4d8e-99d5-2d59957ec339 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:05.982673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e6a1d134-c627-4d8e-99d5-2d59957ec339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:06.002157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:06.085647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a1d134-c627-4d8e-99d5-2d59957ec339 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:06.160348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92180185-8b91-4d08-bd81-be8e5cb802c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:06.160955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92180185-8b91-4d08-bd81-be8e5cb802c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:06.161232 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92180185-8b91-4d08-bd81-be8e5cb802c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:06.161501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92180185-8b91-4d08-bd81-be8e5cb802c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:06.163147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-92180185-8b91-4d08-bd81-be8e5cb802c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:06.235997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:06.274490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92180185-8b91-4d08-bd81-be8e5cb802c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:06.275924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92180185-8b91-4d08-bd81-be8e5cb802c0 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:06.294449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 654/1323] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:08.545358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a5bc1-ac4a-4fdc-9b12-22fd96e6dff0 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:08.546034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a5bc1-ac4a-4fdc-9b12-22fd96e6dff0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:08.546034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a5bc1-ac4a-4fdc-9b12-22fd96e6dff0 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:08.547277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 670/1324] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:08.552687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039784-c66a-4b1f-aff5-a93608af8d40 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:08.553102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039784-c66a-4b1f-aff5-a93608af8d40 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:08.553227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5039784-c66a-4b1f-aff5-a93608af8d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:08.618540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c5039784-c66a-4b1f-aff5-a93608af8d40 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:08.619743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c5039784-c66a-4b1f-aff5-a93608af8d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:08.724537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 655/1325] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:09.008908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a5bc1-ac4a-4fdc-9b12-22fd96e6dff0 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:09.010476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8c8a5bc1-ac4a-4fdc-9b12-22fd96e6dff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:09.112672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a5bc1-ac4a-4fdc-9b12-22fd96e6dff0 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:09.187209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacaf9e-ae3a-4f38-899c-9de60b57f4d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:09.187813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacaf9e-ae3a-4f38-899c-9de60b57f4d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:09.188091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacaf9e-ae3a-4f38-899c-9de60b57f4d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:09.188362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacaf9e-ae3a-4f38-899c-9de60b57f4d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:09.189972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2dacaf9e-ae3a-4f38-899c-9de60b57f4d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:09.207370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:09.300818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacaf9e-ae3a-4f38-899c-9de60b57f4d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:09.302167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacaf9e-ae3a-4f38-899c-9de60b57f4d8 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:09.320955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 671/1326] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:13:11.554619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dc9b8-915e-4301-99b7-8fd899ddb766 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:11.555128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dc9b8-915e-4301-99b7-8fd899ddb766 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:11.555128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dc9b8-915e-4301-99b7-8fd899ddb766 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:11.556440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 656/1327] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:11.561578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7288d8-33e6-4da4-b458-b2e212f4fd2f None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:11.561973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7288d8-33e6-4da4-b458-b2e212f4fd2f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:11.562122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7288d8-33e6-4da4-b458-b2e212f4fd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:11.634324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5a7288d8-33e6-4da4-b458-b2e212f4fd2f None None] MFA Rules not processed for user `8f335b1a4ae84353bed712f1faf9aad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:11.635548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5a7288d8-33e6-4da4-b458-b2e212f4fd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:11.699297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:11.731882 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 672/1328] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:11 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:12.025466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dc9b8-915e-4301-99b7-8fd899ddb766 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:12.027100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-315dc9b8-915e-4301-99b7-8fd899ddb766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:12.067591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:12.144804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dc9b8-915e-4301-99b7-8fd899ddb766 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:12.222651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6cd7f8-4f29-4675-8867-e06c4171f27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:12.223259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd7f8-4f29-4675-8867-e06c4171f27a service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:12.223529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd7f8-4f29-4675-8867-e06c4171f27a service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:12.223794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6cd7f8-4f29-4675-8867-e06c4171f27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:12.225370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fe6cd7f8-4f29-4675-8867-e06c4171f27a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:12.309474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:12.342140 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd7f8-4f29-4675-8867-e06c4171f27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:12.343600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6cd7f8-4f29-4675-8867-e06c4171f27a service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:12.364134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 657/1329] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:14.975847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cced1-d69b-400b-9a99-fdf290d44a29 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:14.976418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cced1-d69b-400b-9a99-fdf290d44a29 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:14.976418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cced1-d69b-400b-9a99-fdf290d44a29 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:14.977753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 673/1330] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:13:14.983605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e48e8d-1f1e-4c6c-8ad8-366b86e6c917 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:14.984015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e48e8d-1f1e-4c6c-8ad8-366b86e6c917 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:14.984209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e48e8d-1f1e-4c6c-8ad8-366b86e6c917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:15.049577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-28e48e8d-1f1e-4c6c-8ad8-366b86e6c917 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:15.050813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-28e48e8d-1f1e-4c6c-8ad8-366b86e6c917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:15.145033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 658/1331] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:15.437770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cced1-d69b-400b-9a99-fdf290d44a29 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:15.439344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5a4cced1-d69b-400b-9a99-fdf290d44a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:15.544780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cced1-d69b-400b-9a99-fdf290d44a29 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:15.619443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93453084-6b27-46cd-a013-b04681ca725d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:15.620038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453084-6b27-46cd-a013-b04681ca725d service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:15.621498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453084-6b27-46cd-a013-b04681ca725d service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:15.621773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453084-6b27-46cd-a013-b04681ca725d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:15.623390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-93453084-6b27-46cd-a013-b04681ca725d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:15.649586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:15.731282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453084-6b27-46cd-a013-b04681ca725d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:15.732617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453084-6b27-46cd-a013-b04681ca725d service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:15.751183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 674/1332] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:17.986594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a73528-d3be-4ab2-bae3-bca18c665f81 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:17.987133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a73528-d3be-4ab2-bae3-bca18c665f81 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:17.987133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a73528-d3be-4ab2-bae3-bca18c665f81 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:17.988476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 659/1333] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:17.993604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7289-cf26-4a6f-9aa0-3426b75d5fa0 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:17.994053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7289-cf26-4a6f-9aa0-3426b75d5fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:17.994158 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a7289-cf26-4a6f-9aa0-3426b75d5fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:18.062925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e73a7289-cf26-4a6f-9aa0-3426b75d5fa0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:18.064132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e73a7289-cf26-4a6f-9aa0-3426b75d5fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:18.137507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:18.159088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 675/1334] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:18.444068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a73528-d3be-4ab2-bae3-bca18c665f81 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:18.445604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-86a73528-d3be-4ab2-bae3-bca18c665f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:18.486214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:18.548641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a73528-d3be-4ab2-bae3-bca18c665f81 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:18.621423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be93b2-89f4-4e5f-8ac7-dbdfda7019a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:18.622022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be93b2-89f4-4e5f-8ac7-dbdfda7019a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:18.622298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be93b2-89f4-4e5f-8ac7-dbdfda7019a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:18.622601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be93b2-89f4-4e5f-8ac7-dbdfda7019a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:18.624179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b7be93b2-89f4-4e5f-8ac7-dbdfda7019a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:18.706089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:18.731479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be93b2-89f4-4e5f-8ac7-dbdfda7019a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:18.732908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be93b2-89f4-4e5f-8ac7-dbdfda7019a5 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:18.751019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 660/1335] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:21.536603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc013848-3333-4458-b96b-2b8f48f7558a None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:21.537169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc013848-3333-4458-b96b-2b8f48f7558a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:21.537169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc013848-3333-4458-b96b-2b8f48f7558a None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:21.538917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 676/1336] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:21.544733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3dce-002f-4329-a539-9707e2e6ba52 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:21.545148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3dce-002f-4329-a539-9707e2e6ba52 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:21.545280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe3dce-002f-4329-a539-9707e2e6ba52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:21.611972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-41fe3dce-002f-4329-a539-9707e2e6ba52 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:21.613181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41fe3dce-002f-4329-a539-9707e2e6ba52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:21.706873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 661/1337] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:21.985335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc013848-3333-4458-b96b-2b8f48f7558a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:21.986894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bc013848-3333-4458-b96b-2b8f48f7558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:22.088664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc013848-3333-4458-b96b-2b8f48f7558a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:22.161187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a4861-b055-46f9-8d04-2cfe0fdcd7d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:22.161787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a4861-b055-46f9-8d04-2cfe0fdcd7d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:22.162064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a4861-b055-46f9-8d04-2cfe0fdcd7d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:22.162368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a4861-b055-46f9-8d04-2cfe0fdcd7d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:22.164073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2d6a4861-b055-46f9-8d04-2cfe0fdcd7d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:22.200229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:22.271847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a4861-b055-46f9-8d04-2cfe0fdcd7d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:22.273193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a4861-b055-46f9-8d04-2cfe0fdcd7d5 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:22.293106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 677/1338] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:26.796124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4756d9-fe4b-4133-a67c-91220a3bf015 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:26.796667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4756d9-fe4b-4133-a67c-91220a3bf015 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:26.796667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4756d9-fe4b-4133-a67c-91220a3bf015 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:26.797997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 662/1339] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:13:26.803202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7efc-ed37-43c5-ab21-7762dc647034 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:26.803616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7efc-ed37-43c5-ab21-7762dc647034 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:26.803739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a7efc-ed37-43c5-ab21-7762dc647034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:26.986873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-914a7efc-ed37-43c5-ab21-7762dc647034 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:26.987990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-914a7efc-ed37-43c5-ab21-7762dc647034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:27.074066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:27.084954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 678/1340] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:27.389223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876be61-3819-4ae0-b320-b40c290ca819 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:27.389529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876be61-3819-4ae0-b320-b40c290ca819 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:27.389825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876be61-3819-4ae0-b320-b40c290ca819 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:27.391403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 663/1341] 10.222.0.20 () {54 vars in 965 bytes} [Fri Sep 9 23:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:27.397263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af82cd-0523-4ade-8702-1f5f66357156 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:27.397549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af82cd-0523-4ade-8702-1f5f66357156 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:27.397823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af82cd-0523-4ade-8702-1f5f66357156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:27.477497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-57af82cd-0523-4ade-8702-1f5f66357156 None None] MFA Rules not processed for user `a9da2d7ef80f4622b58c9041b65df992`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:27.478745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-57af82cd-0523-4ade-8702-1f5f66357156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:27.559527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 679/1342] 10.222.0.20 () {58 vars in 1062 bytes} [Fri Sep 9 23:13:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:27.568588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876be61-3819-4ae0-b320-b40c290ca819 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:27.570138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7876be61-3819-4ae0-b320-b40c290ca819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:27.618944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:27.675795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876be61-3819-4ae0-b320-b40c290ca819 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:27.755096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e977c-0692-43ff-8fe7-0df1cfd6a4ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:27.755792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e977c-0692-43ff-8fe7-0df1cfd6a4ff service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:27.756075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e977c-0692-43ff-8fe7-0df1cfd6a4ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:27.756547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e977c-0692-43ff-8fe7-0df1cfd6a4ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:27.758449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a37e977c-0692-43ff-8fe7-0df1cfd6a4ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:27.856195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:27.875548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e977c-0692-43ff-8fe7-0df1cfd6a4ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:27.876921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e977c-0692-43ff-8fe7-0df1cfd6a4ff service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:27.897664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 664/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:31.704617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae388d-9591-46ae-bdb6-67a873417ab9 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:31.705174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae388d-9591-46ae-bdb6-67a873417ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:31.705174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae388d-9591-46ae-bdb6-67a873417ab9 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:31.706512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 680/1344] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:31.711807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f6b3-804c-400a-828b-d51e8cb8ecf7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:31.712453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f6b3-804c-400a-828b-d51e8cb8ecf7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:31.712453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f6b3-804c-400a-828b-d51e8cb8ecf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:31.783684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-76e3f6b3-804c-400a-828b-d51e8cb8ecf7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:31.784913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-76e3f6b3-804c-400a-828b-d51e8cb8ecf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:31.887606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 665/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:13:32.176535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae388d-9591-46ae-bdb6-67a873417ab9 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:32.178056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-40ae388d-9591-46ae-bdb6-67a873417ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:32.271789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:32.288637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae388d-9591-46ae-bdb6-67a873417ab9 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:32.369344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94563c4b-dd13-4eb1-b859-9e235c675305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:32.369344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563c4b-dd13-4eb1-b859-9e235c675305 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:32.369344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563c4b-dd13-4eb1-b859-9e235c675305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:32.369344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563c4b-dd13-4eb1-b859-9e235c675305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:32.370764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-94563c4b-dd13-4eb1-b859-9e235c675305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:32.488739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94563c4b-dd13-4eb1-b859-9e235c675305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:32.490210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94563c4b-dd13-4eb1-b859-9e235c675305 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:32.509574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 681/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:37.028225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64088e78-1197-493e-8ad2-4193a81a629c None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:37.028779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64088e78-1197-493e-8ad2-4193a81a629c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:37.028779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64088e78-1197-493e-8ad2-4193a81a629c None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:37.030290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 666/1347] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:37.035433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbbed4-0b42-4901-b0c8-dd504cdf333f None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:37.035873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbbed4-0b42-4901-b0c8-dd504cdf333f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:37.035971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbbed4-0b42-4901-b0c8-dd504cdf333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:37.052142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:37.106254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-1dbbbed4-0b42-4901-b0c8-dd504cdf333f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:37.107489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1dbbbed4-0b42-4901-b0c8-dd504cdf333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:37.211235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 682/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:37.490102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64088e78-1197-493e-8ad2-4193a81a629c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:37.491682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-64088e78-1197-493e-8ad2-4193a81a629c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:37.549126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:37.594415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64088e78-1197-493e-8ad2-4193a81a629c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:37.675420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31762962-6e30-43ef-9acc-675a5df93f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:37.676018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762962-6e30-43ef-9acc-675a5df93f0f service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:37.676294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762962-6e30-43ef-9acc-675a5df93f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:37.676561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31762962-6e30-43ef-9acc-675a5df93f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:37.678137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-31762962-6e30-43ef-9acc-675a5df93f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:37.783835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:37.787274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31762962-6e30-43ef-9acc-675a5df93f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:37.788716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31762962-6e30-43ef-9acc-675a5df93f0f service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:37.922433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 667/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:42.940006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73ee39-9b62-4be2-8556-63887be5843a None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:42.940553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73ee39-9b62-4be2-8556-63887be5843a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:42.940553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73ee39-9b62-4be2-8556-63887be5843a None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:42.942278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 683/1350] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:42.949171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1306-ffec-44a0-873d-d5bbcca981a9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:42.949171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1306-ffec-44a0-873d-d5bbcca981a9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:42.949636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f1306-ffec-44a0-873d-d5bbcca981a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:43.101188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fa7f1306-ffec-44a0-873d-d5bbcca981a9 None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:43.103092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fa7f1306-ffec-44a0-873d-d5bbcca981a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:43.239829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 668/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:13:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:47.209612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=59 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:47.414887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005345c5-cbe1-449c-badb-e4bf423c7ce3 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:47.414887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005345c5-cbe1-449c-badb-e4bf423c7ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:47.414887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005345c5-cbe1-449c-badb-e4bf423c7ce3 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:47.414887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 684/1352] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:47.420317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8671fea-14dd-4336-97e9-85ac712a2b84 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:47.421181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8671fea-14dd-4336-97e9-85ac712a2b84 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:47.421597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8671fea-14dd-4336-97e9-85ac712a2b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:47.484360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:47.507520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b8671fea-14dd-4336-97e9-85ac712a2b84 None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:47.508716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b8671fea-14dd-4336-97e9-85ac712a2b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:47.619388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 669/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:13:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:48.941828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40cf8a-a177-4e93-a144-601abafc3a37 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:48.942528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40cf8a-a177-4e93-a144-601abafc3a37 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:48.942528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40cf8a-a177-4e93-a144-601abafc3a37 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:48.943903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 685/1354] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:48.950738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318f7cb-0172-47f0-bc43-3f6522f69e12 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:48.951210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318f7cb-0172-47f0-bc43-3f6522f69e12 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:48.951293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318f7cb-0172-47f0-bc43-3f6522f69e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:49.030167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0318f7cb-0172-47f0-bc43-3f6522f69e12 None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:49.032943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0318f7cb-0172-47f0-bc43-3f6522f69e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:49.079126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:49.146360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 670/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:13:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:13:52.841044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9c31e-3a9d-4bdb-a60e-0d6946dbffbc None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:52.841569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9c31e-3a9d-4bdb-a60e-0d6946dbffbc None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:52.841569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9c31e-3a9d-4bdb-a60e-0d6946dbffbc None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:52.843320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 686/1356] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:52.848768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a8843-c848-436d-944b-82452a8f83a7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:52.849196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a8843-c848-436d-944b-82452a8f83a7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:52.849296 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a8843-c848-436d-944b-82452a8f83a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:52.923814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-717a8843-c848-436d-944b-82452a8f83a7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:52.925361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-717a8843-c848-436d-944b-82452a8f83a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:53.022615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 671/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:53.331337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a67a79-a0ed-4f45-ade9-3f9d40ffbf2a None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:53.332061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a67a79-a0ed-4f45-ade9-3f9d40ffbf2a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:53.332557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a67a79-a0ed-4f45-ade9-3f9d40ffbf2a None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:53.334601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 687/1358] 10.222.0.20 () {54 vars in 962 bytes} [Fri Sep 9 23:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:53.342153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639a508-1f0b-4cbe-ad8c-e8fb8332b0c0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:53.343066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639a508-1f0b-4cbe-ad8c-e8fb8332b0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:53.344288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639a508-1f0b-4cbe-ad8c-e8fb8332b0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:53.363785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:53.436423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7639a508-1f0b-4cbe-ad8c-e8fb8332b0c0 None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:53.437610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7639a508-1f0b-4cbe-ad8c-e8fb8332b0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:53.561046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 672/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Sep 9 23:13:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:53.568629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a67a79-a0ed-4f45-ade9-3f9d40ffbf2a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:53.570267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-25a67a79-a0ed-4f45-ade9-3f9d40ffbf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:53.726576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a67a79-a0ed-4f45-ade9-3f9d40ffbf2a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:53.828659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:53.872052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae3e98-6cec-4976-a264-d4045892be11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:53.872887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3e98-6cec-4976-a264-d4045892be11 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:53.873165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3e98-6cec-4976-a264-d4045892be11 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:53.873392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae3e98-6cec-4976-a264-d4045892be11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:53.875105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-50ae3e98-6cec-4976-a264-d4045892be11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:53.977273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae3e98-6cec-4976-a264-d4045892be11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:53.978910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae3e98-6cec-4976-a264-d4045892be11 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:53.999831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 688/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:13:56.452934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094961f-e47e-4db6-ab47-783e6953f6f0 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:56.453544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094961f-e47e-4db6-ab47-783e6953f6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:56.453544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094961f-e47e-4db6-ab47-783e6953f6f0 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:56.454929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 673/1361] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:13:56.466894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fa019-3ab0-4208-9e91-b9124fa56e8d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:56.466894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fa019-3ab0-4208-9e91-b9124fa56e8d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:56.466894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248fa019-3ab0-4208-9e91-b9124fa56e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:56.534367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-248fa019-3ab0-4208-9e91-b9124fa56e8d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:13:56.535642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-248fa019-3ab0-4208-9e91-b9124fa56e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:56.576600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:56.639068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 689/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:13:56.937491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094961f-e47e-4db6-ab47-783e6953f6f0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:13:56.939738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8094961f-e47e-4db6-ab47-783e6953f6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:57.025570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:57.128018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094961f-e47e-4db6-ab47-783e6953f6f0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:13:57.264319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6633f1f-1b52-4ea9-857d-7c7026ce5969 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:13:57.264922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633f1f-1b52-4ea9-857d-7c7026ce5969 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:13:57.265203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633f1f-1b52-4ea9-857d-7c7026ce5969 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:13:57.265639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6633f1f-1b52-4ea9-857d-7c7026ce5969 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:13:57.267319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e6633f1f-1b52-4ea9-857d-7c7026ce5969 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:13:57.291054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:13:57.365171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6633f1f-1b52-4ea9-857d-7c7026ce5969 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:13:57.366732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6633f1f-1b52-4ea9-857d-7c7026ce5969 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:13:57.385261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 674/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:00.022077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770128c5-3182-4b73-bd8e-d13a14224efa None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:00.022670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770128c5-3182-4b73-bd8e-d13a14224efa None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:00.022670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770128c5-3182-4b73-bd8e-d13a14224efa None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:00.024160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 690/1364] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:00.029396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cccdb-773c-444f-b773-73fd36295748 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:00.029830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cccdb-773c-444f-b773-73fd36295748 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:00.030002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cccdb-773c-444f-b773-73fd36295748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:00.102617 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-537cccdb-773c-444f-b773-73fd36295748 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:00.103819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-537cccdb-773c-444f-b773-73fd36295748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:00.207217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:00.214697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 675/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:00.515259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ca941-f60a-4e7b-8d9f-84a8535da346 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:00.515493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ca941-f60a-4e7b-8d9f-84a8535da346 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:00.515953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ca941-f60a-4e7b-8d9f-84a8535da346 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:00.517692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 691/1366] 10.222.0.20 () {54 vars in 962 bytes} [Fri Sep 9 23:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:00.523462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da243b4-850b-4f5b-b069-663c61a2d022 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:00.523929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da243b4-850b-4f5b-b069-663c61a2d022 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:00.528428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da243b4-850b-4f5b-b069-663c61a2d022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:00.605360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3da243b4-850b-4f5b-b069-663c61a2d022 None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:00.606845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3da243b4-850b-4f5b-b069-663c61a2d022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:00.738105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 676/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Sep 9 23:14:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:00.742024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ca941-f60a-4e7b-8d9f-84a8535da346 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:00.743586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-116ca941-f60a-4e7b-8d9f-84a8535da346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:00.902210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ca941-f60a-4e7b-8d9f-84a8535da346 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:00.922552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:01.032043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986bcf63-3904-4131-a0ca-8402b76cb541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:01.033465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bcf63-3904-4131-a0ca-8402b76cb541 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:01.034039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bcf63-3904-4131-a0ca-8402b76cb541 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:01.034529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986bcf63-3904-4131-a0ca-8402b76cb541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:01.036630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-986bcf63-3904-4131-a0ca-8402b76cb541 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:01.082882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e63bb-7892-40f9-8c1c-46081b48b50e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:01.083927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e63bb-7892-40f9-8c1c-46081b48b50e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:01.084204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e63bb-7892-40f9-8c1c-46081b48b50e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:01.085949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 677/1368] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:14:01.091500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da12b-bcae-4d79-b544-6d252cc9b25a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:01.091778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da12b-bcae-4d79-b544-6d252cc9b25a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:01.092046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625da12b-bcae-4d79-b544-6d252cc9b25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:01.137584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bcf63-3904-4131-a0ca-8402b76cb541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:01.139234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986bcf63-3904-4131-a0ca-8402b76cb541 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:01.156395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:01.164633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 692/1369] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:01.165309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-625da12b-bcae-4d79-b544-6d252cc9b25a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:01.167342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-625da12b-bcae-4d79-b544-6d252cc9b25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:01.270024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 678/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:01.608691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fc4c-8d0c-47f9-8d6b-8f506c3e3c80 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:01.608691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fc4c-8d0c-47f9-8d6b-8f506c3e3c80 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:01.608691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fc4c-8d0c-47f9-8d6b-8f506c3e3c80 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:01.610046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 693/1371] 10.222.0.20 () {54 vars in 964 bytes} [Fri Sep 9 23:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:01.616151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47d1f-5168-490f-a11a-e039d83fbbeb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:01.616691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47d1f-5168-490f-a11a-e039d83fbbeb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:01.617143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47d1f-5168-490f-a11a-e039d83fbbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:01.702006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d0b47d1f-5168-490f-a11a-e039d83fbbeb None None] MFA Rules not processed for user `c54fab93990146e0bb746abfd0843444`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:01.703636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d0b47d1f-5168-490f-a11a-e039d83fbbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:01.778626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:01.788727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 679/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Fri Sep 9 23:14:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:01.796949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5fc4c-8d0c-47f9-8d6b-8f506c3e3c80 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:01.798556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0ed5fc4c-8d0c-47f9-8d6b-8f506c3e3c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:01.830320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:01.917579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5fc4c-8d0c-47f9-8d6b-8f506c3e3c80 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:01.997302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7c673-35b8-4ace-a87f-b52b08a13350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:01.998208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c673-35b8-4ace-a87f-b52b08a13350 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:01.998515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c673-35b8-4ace-a87f-b52b08a13350 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:01.998790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7c673-35b8-4ace-a87f-b52b08a13350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:02.000376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6bc7c673-35b8-4ace-a87f-b52b08a13350 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:02.087216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:02.129685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7c673-35b8-4ace-a87f-b52b08a13350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:02.131156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc7c673-35b8-4ace-a87f-b52b08a13350 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:02.156388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 694/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:03.613749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8063-1767-45dd-9214-baa582197400 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:03.614238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8063-1767-45dd-9214-baa582197400 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:03.614370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab8063-1767-45dd-9214-baa582197400 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:03.615726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 680/1374] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:03.622044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83eb9-e92c-4822-80b1-077d1126590d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:03.622514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83eb9-e92c-4822-80b1-077d1126590d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:03.622611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd83eb9-e92c-4822-80b1-077d1126590d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:03.689097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-cfd83eb9-e92c-4822-80b1-077d1126590d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:03.690239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cfd83eb9-e92c-4822-80b1-077d1126590d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:03.786915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 695/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:04.079540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab8063-1767-45dd-9214-baa582197400 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:04.081054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3aab8063-1767-45dd-9214-baa582197400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:04.234612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab8063-1767-45dd-9214-baa582197400 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:04.306256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:04.358922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380667c6-1745-405d-9bde-4dd2c6d2c591 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:04.359530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380667c6-1745-405d-9bde-4dd2c6d2c591 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:04.359802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380667c6-1745-405d-9bde-4dd2c6d2c591 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:04.360069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380667c6-1745-405d-9bde-4dd2c6d2c591 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:04.361645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-380667c6-1745-405d-9bde-4dd2c6d2c591 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:04.471704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380667c6-1745-405d-9bde-4dd2c6d2c591 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:04.473306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380667c6-1745-405d-9bde-4dd2c6d2c591 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:04.496101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 681/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:06.865708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c1e9d-b677-4e2f-8a52-29e26b61a3db None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:06.866240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c1e9d-b677-4e2f-8a52-29e26b61a3db None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:06.866240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c1e9d-b677-4e2f-8a52-29e26b61a3db None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:06.867741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 696/1377] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:06.873411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadff7b-d26e-475e-be02-3c713eed325c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:06.873844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadff7b-d26e-475e-be02-3c713eed325c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:06.874012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadff7b-d26e-475e-be02-3c713eed325c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:06.939347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2cadff7b-d26e-475e-be02-3c713eed325c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:06.940527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2cadff7b-d26e-475e-be02-3c713eed325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:06.971486 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:07.039499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 682/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:07.330751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c1e9d-b677-4e2f-8a52-29e26b61a3db None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:07.332454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-259c1e9d-b677-4e2f-8a52-29e26b61a3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:07.374974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:07.500215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c1e9d-b677-4e2f-8a52-29e26b61a3db None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:07.616897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:07.622466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83a444-a928-4035-8117-9f80a7a8fdcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:07.623139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83a444-a928-4035-8117-9f80a7a8fdcd service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:07.623488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83a444-a928-4035-8117-9f80a7a8fdcd service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:07.623810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83a444-a928-4035-8117-9f80a7a8fdcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:07.625562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9d83a444-a928-4035-8117-9f80a7a8fdcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:07.720963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83a444-a928-4035-8117-9f80a7a8fdcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:07.722651 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83a444-a928-4035-8117-9f80a7a8fdcd service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:07.741402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 697/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:10.172801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9399292-fa93-4e0d-ac35-b4d0e08f67c6 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:10.173386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9399292-fa93-4e0d-ac35-b4d0e08f67c6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:10.173494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9399292-fa93-4e0d-ac35-b4d0e08f67c6 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:10.175214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 683/1380] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:10.181460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3c3a7-a150-4bcc-82c2-6bdefc4cec5e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:10.182445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3c3a7-a150-4bcc-82c2-6bdefc4cec5e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:10.182617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3c3a7-a150-4bcc-82c2-6bdefc4cec5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:10.252892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-07f3c3a7-a150-4bcc-82c2-6bdefc4cec5e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:10.254139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-07f3c3a7-a150-4bcc-82c2-6bdefc4cec5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:10.321879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:10.364109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 698/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:10.668527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9399292-fa93-4e0d-ac35-b4d0e08f67c6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:10.670163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d9399292-fa93-4e0d-ac35-b4d0e08f67c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:10.821872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9399292-fa93-4e0d-ac35-b4d0e08f67c6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:10.835388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:10.939915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e24c5-1695-49d8-b612-e45519216461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:10.940435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e24c5-1695-49d8-b612-e45519216461 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:10.940702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e24c5-1695-49d8-b612-e45519216461 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:10.940962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e24c5-1695-49d8-b612-e45519216461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:10.942566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a50e24c5-1695-49d8-b612-e45519216461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:11.033844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e24c5-1695-49d8-b612-e45519216461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:11.035542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e24c5-1695-49d8-b612-e45519216461 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:11.054164 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 684/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:13.727145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b2c2b-b06c-4e61-847a-54e33a29eddb None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:13.727659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b2c2b-b06c-4e61-847a-54e33a29eddb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:13.727772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b2c2b-b06c-4e61-847a-54e33a29eddb None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:13.729583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 699/1383] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:13.735406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2cd50-0129-4b1f-860b-56b8892ed1ce None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:13.735976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2cd50-0129-4b1f-860b-56b8892ed1ce None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:13.735976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2cd50-0129-4b1f-860b-56b8892ed1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:13.758566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:13.809078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-93a2cd50-0129-4b1f-860b-56b8892ed1ce None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:13.810961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-93a2cd50-0129-4b1f-860b-56b8892ed1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:13.943086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 685/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:14.232348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b2c2b-b06c-4e61-847a-54e33a29eddb None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:14.234292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fa0b2c2b-b06c-4e61-847a-54e33a29eddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:14.405181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b2c2b-b06c-4e61-847a-54e33a29eddb None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:14.464569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:14.527952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4717f05d-8161-41a1-9dfa-a5d88f7adadd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:14.528552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717f05d-8161-41a1-9dfa-a5d88f7adadd service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:14.528822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717f05d-8161-41a1-9dfa-a5d88f7adadd service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:14.529129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4717f05d-8161-41a1-9dfa-a5d88f7adadd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:14.530712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4717f05d-8161-41a1-9dfa-a5d88f7adadd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:14.632953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717f05d-8161-41a1-9dfa-a5d88f7adadd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:14.634725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4717f05d-8161-41a1-9dfa-a5d88f7adadd service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:14.655598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 700/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:16.998800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e446-b5f9-453a-ad86-c316fdf7c29b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:16.999406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e446-b5f9-453a-ad86-c316fdf7c29b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:16.999406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e446-b5f9-453a-ad86-c316fdf7c29b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:17.000883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 686/1386] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:17.005963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579137c-8f8d-4fad-9138-7fa97b1d7631 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:17.006458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579137c-8f8d-4fad-9138-7fa97b1d7631 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:17.006681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579137c-8f8d-4fad-9138-7fa97b1d7631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:17.073929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0579137c-8f8d-4fad-9138-7fa97b1d7631 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:17.075168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0579137c-8f8d-4fad-9138-7fa97b1d7631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:17.084029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:17.174445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 701/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:17.483363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9e446-b5f9-453a-ad86-c316fdf7c29b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:17.485177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8ca9e446-b5f9-453a-ad86-c316fdf7c29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:17.589686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:17.644740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9e446-b5f9-453a-ad86-c316fdf7c29b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:17.768747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaf8a0-5610-4a87-8597-38147980e8e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:17.769348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf8a0-5610-4a87-8597-38147980e8e3 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:17.769620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf8a0-5610-4a87-8597-38147980e8e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:17.769886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf8a0-5610-4a87-8597-38147980e8e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:17.771500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e3eaf8a0-5610-4a87-8597-38147980e8e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:17.783828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20805ad-4c28-4bf0-b6c4-2b3bbd59ece7 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:17.784121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20805ad-4c28-4bf0-b6c4-2b3bbd59ece7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:17.784427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20805ad-4c28-4bf0-b6c4-2b3bbd59ece7 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:17.785853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 702/1388] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:14:17.791062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193621-81b4-4187-aab3-2b22daf79bfc None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:17.791915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193621-81b4-4187-aab3-2b22daf79bfc None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:17.792193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193621-81b4-4187-aab3-2b22daf79bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:17.834083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:17.859609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-b1193621-81b4-4187-aab3-2b22daf79bfc None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:17.860734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b1193621-81b4-4187-aab3-2b22daf79bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:17.865291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaf8a0-5610-4a87-8597-38147980e8e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:17.866905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaf8a0-5610-4a87-8597-38147980e8e3 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:17.891627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 687/1389] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:17.960455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:17.974370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 703/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:18.286876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69697-42e7-423c-b6d4-569bb6b0d7a7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:18.287342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69697-42e7-423c-b6d4-569bb6b0d7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:18.287576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b69697-42e7-423c-b6d4-569bb6b0d7a7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:18.289026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 688/1391] 10.222.0.20 () {54 vars in 966 bytes} [Fri Sep 9 23:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:18.294711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1700e-7e9c-41e2-bcdd-bc0c3cc7e7f9 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:18.295167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1700e-7e9c-41e2-bcdd-bc0c3cc7e7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:18.295377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1700e-7e9c-41e2-bcdd-bc0c3cc7e7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:18.372987 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-33d1700e-7e9c-41e2-bcdd-bc0c3cc7e7f9 None None] MFA Rules not processed for user `c1842c116caa4a5d9d75c8605eeec0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:18.374186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-33d1700e-7e9c-41e2-bcdd-bc0c3cc7e7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:18.455992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 704/1392] 10.222.0.20 () {58 vars in 1063 bytes} [Fri Sep 9 23:14:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:18.465151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b69697-42e7-423c-b6d4-569bb6b0d7a7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:18.466816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-14b69697-42e7-423c-b6d4-569bb6b0d7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:18.575476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b69697-42e7-423c-b6d4-569bb6b0d7a7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:18.649786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3615335-f2e1-48d2-a9f2-0b47bc5b4db2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1842c116caa4a5d9d75c8605eeec0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:18.650443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3615335-f2e1-48d2-a9f2-0b47bc5b4db2 service glance] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:18.651293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3615335-f2e1-48d2-a9f2-0b47bc5b4db2 service glance] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:18.651557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3615335-f2e1-48d2-a9f2-0b47bc5b4db2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:18.653152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f3615335-f2e1-48d2-a9f2-0b47bc5b4db2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:18.662046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:18.763017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3615335-f2e1-48d2-a9f2-0b47bc5b4db2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:18.764421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3615335-f2e1-48d2-a9f2-0b47bc5b4db2 service glance] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:18.782659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 689/1393] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:19.681602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22764874-6979-4d88-bc09-84474ee2eb9d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:19.682250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22764874-6979-4d88-bc09-84474ee2eb9d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:19.682388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22764874-6979-4d88-bc09-84474ee2eb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:19.751443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-22764874-6979-4d88-bc09-84474ee2eb9d None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:19.752539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-22764874-6979-4d88-bc09-84474ee2eb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:19.787086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:19.837401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 705/1394] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:14:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:19.888707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e220e3-3aed-4249-a5e9-a4846fc96aa1 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:19.889225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e220e3-3aed-4249-a5e9-a4846fc96aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:19.889500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e220e3-3aed-4249-a5e9-a4846fc96aa1 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:19.890963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 690/1395] 10.222.0.20 () {54 vars in 943 bytes} [Fri Sep 9 23:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:19.897400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ea59-d3d4-4527-8e91-8b9cb12c4d65 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:19.897857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ea59-d3d4-4527-8e91-8b9cb12c4d65 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:19.898131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8ea59-d3d4-4527-8e91-8b9cb12c4d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:19.975996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-dac8ea59-d3d4-4527-8e91-8b9cb12c4d65 None None] MFA Rules not processed for user `135696b09c2c4f809c49e0af200f32de`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:19.977187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-dac8ea59-d3d4-4527-8e91-8b9cb12c4d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:20.086369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 706/1396] 10.222.0.20 () {58 vars in 1040 bytes} [Fri Sep 9 23:14:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:20.096300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e220e3-3aed-4249-a5e9-a4846fc96aa1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:20.097853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-56e220e3-3aed-4249-a5e9-a4846fc96aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:20.215331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:20.244146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e220e3-3aed-4249-a5e9-a4846fc96aa1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:20.309203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959277f-7cc3-4dc8-bd28-eb06f5c2e619 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:20.309489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959277f-7cc3-4dc8-bd28-eb06f5c2e619 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:20.309762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959277f-7cc3-4dc8-bd28-eb06f5c2e619 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:20.311328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 707/1397] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:20.316032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442052-6362-491e-a79b-3b9fa3515e63 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:20.316309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442052-6362-491e-a79b-3b9fa3515e63 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:20.316582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92442052-6362-491e-a79b-3b9fa3515e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:20.326713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:20.351573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:20.352168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:20.352439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:20.352703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:20.354863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:20.385817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-92442052-6362-491e-a79b-3b9fa3515e63 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:20.387057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-92442052-6362-491e-a79b-3b9fa3515e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:20.449670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:20.451281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:20.459454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 691/1398] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:14:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:14:20.491098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 708/1399] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:14:20.786069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:20.787844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:20.801854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:20.907723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b382376-8aa5-4895-b58e-28c0567d5416 service swift] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:21.057332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea582d-c45f-4434-a9ee-850c12ea3023 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:21.057878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea582d-c45f-4434-a9ee-850c12ea3023 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:21.058160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea582d-c45f-4434-a9ee-850c12ea3023 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:21.058465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea582d-c45f-4434-a9ee-850c12ea3023 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:21.060095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-79ea582d-c45f-4434-a9ee-850c12ea3023 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:21.108653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:21.178151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea582d-c45f-4434-a9ee-850c12ea3023 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:21.180361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea582d-c45f-4434-a9ee-850c12ea3023 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:21.225414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 692/1400] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:24.360105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186c19-819a-428e-953e-4bb392d7cdbc None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:24.360683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186c19-819a-428e-953e-4bb392d7cdbc None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:24.360683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186c19-819a-428e-953e-4bb392d7cdbc None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:24.362115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 709/1401] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:24.370045 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a962a3-efbe-4a39-8f0b-fed9d77c6f75 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:24.370594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a962a3-efbe-4a39-8f0b-fed9d77c6f75 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:24.370826 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a962a3-efbe-4a39-8f0b-fed9d77c6f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:24.453977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b4a962a3-efbe-4a39-8f0b-fed9d77c6f75 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:24.455248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b4a962a3-efbe-4a39-8f0b-fed9d77c6f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:24.567532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 693/1402] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:24.862749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af186c19-819a-428e-953e-4bb392d7cdbc None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:24.864494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-af186c19-819a-428e-953e-4bb392d7cdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:24.965139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:25.037662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af186c19-819a-428e-953e-4bb392d7cdbc None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:25.162705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d28fc-ba98-4732-9bb4-f932c6b34701 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:25.163238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d28fc-ba98-4732-9bb4-f932c6b34701 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:25.163544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d28fc-ba98-4732-9bb4-f932c6b34701 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:25.163784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d28fc-ba98-4732-9bb4-f932c6b34701 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:25.165388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e37d28fc-ba98-4732-9bb4-f932c6b34701 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:25.220539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:25.275266 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d28fc-ba98-4732-9bb4-f932c6b34701 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:25.277029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d28fc-ba98-4732-9bb4-f932c6b34701 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:25.295306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 710/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:14:27.747939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856e50-15c6-4a84-9ef5-10c2946fa43d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:27.748453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856e50-15c6-4a84-9ef5-10c2946fa43d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:27.748453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856e50-15c6-4a84-9ef5-10c2946fa43d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:27.750014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 694/1404] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:27.755455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f85fa-cd05-4382-8b84-f5a4d8361d22 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:27.755900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f85fa-cd05-4382-8b84-f5a4d8361d22 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:27.755983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f85fa-cd05-4382-8b84-f5a4d8361d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:27.831268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-278f85fa-cd05-4382-8b84-f5a4d8361d22 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:27.832502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-278f85fa-cd05-4382-8b84-f5a4d8361d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:27.942946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 711/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:28.251442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa856e50-15c6-4a84-9ef5-10c2946fa43d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:28.253150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-aa856e50-15c6-4a84-9ef5-10c2946fa43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:28.262277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:28.441294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa856e50-15c6-4a84-9ef5-10c2946fa43d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:28.539528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:28.574730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371a72b-f67c-484b-86bb-7b872b21ec3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:28.575372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371a72b-f67c-484b-86bb-7b872b21ec3c service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:28.575649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371a72b-f67c-484b-86bb-7b872b21ec3c service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:28.575919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371a72b-f67c-484b-86bb-7b872b21ec3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:28.577531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4371a72b-f67c-484b-86bb-7b872b21ec3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:28.693293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371a72b-f67c-484b-86bb-7b872b21ec3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:28.694905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371a72b-f67c-484b-86bb-7b872b21ec3c service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:28.717416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 695/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:31.044813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e245ab-c215-4197-8ab0-770d00742002 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:31.045397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e245ab-c215-4197-8ab0-770d00742002 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:31.045397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e245ab-c215-4197-8ab0-770d00742002 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:31.046726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 712/1407] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:31.052812 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19421f-762e-4e19-8204-4be6a7accd50 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:31.053335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19421f-762e-4e19-8204-4be6a7accd50 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:31.053335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19421f-762e-4e19-8204-4be6a7accd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:31.145262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2f19421f-762e-4e19-8204-4be6a7accd50 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:31.159107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2f19421f-762e-4e19-8204-4be6a7accd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:31.217214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:31.297777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 696/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:31.589257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e245ab-c215-4197-8ab0-770d00742002 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:31.591098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-85e245ab-c215-4197-8ab0-770d00742002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:31.623928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:31.761696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e245ab-c215-4197-8ab0-770d00742002 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:31.870741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:31.887732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd2df0-42bf-4370-93a3-d80ce6e14513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:31.888350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd2df0-42bf-4370-93a3-d80ce6e14513 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:31.888629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd2df0-42bf-4370-93a3-d80ce6e14513 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:31.888905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd2df0-42bf-4370-93a3-d80ce6e14513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:31.890687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4ddd2df0-42bf-4370-93a3-d80ce6e14513 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:32.011107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd2df0-42bf-4370-93a3-d80ce6e14513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:32.012833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd2df0-42bf-4370-93a3-d80ce6e14513 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:32.059973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 713/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:41.296082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=24 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:42.044256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=51 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:44.310686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22218474-01be-4eb8-8aca-618813f8dd17 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:44.311267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22218474-01be-4eb8-8aca-618813f8dd17 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:44.311267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22218474-01be-4eb8-8aca-618813f8dd17 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:44.312607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 697/1410] 10.222.0.20 () {54 vars in 918 bytes} [Fri Sep 9 23:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:44.317615 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7c16-c4c3-4807-a798-3eb1dfae82a4 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:44.318071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7c16-c4c3-4807-a798-3eb1dfae82a4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:44.318176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7c16-c4c3-4807-a798-3eb1dfae82a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:44.396239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a40f7c16-c4c3-4807-a798-3eb1dfae82a4 None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:44.397363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a40f7c16-c4c3-4807-a798-3eb1dfae82a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:44.506763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 714/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Sep 9 23:14:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:44.521839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576ca2c-86e8-4d77-90d8-d31017b39b63 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:44.522124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576ca2c-86e8-4d77-90d8-d31017b39b63 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:44.522435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c576ca2c-86e8-4d77-90d8-d31017b39b63 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:44.523780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 698/1412] 10.222.0.20 () {54 vars in 918 bytes} [Fri Sep 9 23:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:44.528666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9eee-4764-4f51-92eb-88ebf2c50e1a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:44.528951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9eee-4764-4f51-92eb-88ebf2c50e1a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:44.529249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9eee-4764-4f51-92eb-88ebf2c50e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:44.602152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:44.604824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5e1a9eee-4764-4f51-92eb-88ebf2c50e1a None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:44.606049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5e1a9eee-4764-4f51-92eb-88ebf2c50e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:44.727343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 715/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Fri Sep 9 23:14:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:47.350971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477e51c-3c63-46c7-9b0a-ab93a0428cd4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.351400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477e51c-3c63-46c7-9b0a-ab93a0428cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.351496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477e51c-3c63-46c7-9b0a-ab93a0428cd4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.352853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:47.358527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f66b77-b8d6-41f5-a244-b9c35c69bd21 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:47.361808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f66b77-b8d6-41f5-a244-b9c35c69bd21 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:47.361808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f66b77-b8d6-41f5-a244-b9c35c69bd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:47.426750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-33f66b77-b8d6-41f5-a244-b9c35c69bd21 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:47.427974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-33f66b77-b8d6-41f5-a244-b9c35c69bd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:47.485874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:47.528082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:47.820673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477e51c-3c63-46c7-9b0a-ab93a0428cd4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:47.822986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0477e51c-3c63-46c7-9b0a-ab93a0428cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:47.969273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477e51c-3c63-46c7-9b0a-ab93a0428cd4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:48.039364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:48.080820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc5729-32c2-476a-95a7-981e24866396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:48.081508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5729-32c2-476a-95a7-981e24866396 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:48.081930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5729-32c2-476a-95a7-981e24866396 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:48.082457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc5729-32c2-476a-95a7-981e24866396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:48.084084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d9fc5729-32c2-476a-95a7-981e24866396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:48.178500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5729-32c2-476a-95a7-981e24866396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:48.180083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc5729-32c2-476a-95a7-981e24866396 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:48.199343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 700/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:49.384956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c64a7c-b598-4cd4-babc-c7812cebafe8 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:49.385519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c64a7c-b598-4cd4-babc-c7812cebafe8 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:49.385519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c64a7c-b598-4cd4-babc-c7812cebafe8 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:49.386789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:49.393091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a8fd3-ef83-42ca-8e21-fc9c56ee67bc None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:49.393534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a8fd3-ef83-42ca-8e21-fc9c56ee67bc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:49.393637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a8fd3-ef83-42ca-8e21-fc9c56ee67bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:49.470221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8d2a8fd3-ef83-42ca-8e21-fc9c56ee67bc None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:49.471449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8d2a8fd3-ef83-42ca-8e21-fc9c56ee67bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:49.498576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:49.566363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:49.722435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c64a7c-b598-4cd4-babc-c7812cebafe8 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:49.724006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c2c64a7c-b598-4cd4-babc-c7812cebafe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:49.870514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c64a7c-b598-4cd4-babc-c7812cebafe8 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:49.911388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:49.985165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8584c-f383-41a8-8f16-6ebd69d3b836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:49.985684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8584c-f383-41a8-8f16-6ebd69d3b836 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:49.985963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8584c-f383-41a8-8f16-6ebd69d3b836 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:49.986230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8584c-f383-41a8-8f16-6ebd69d3b836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:49.987854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-54b8584c-f383-41a8-8f16-6ebd69d3b836 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:50.077753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8584c-f383-41a8-8f16-6ebd69d3b836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:50.079368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8584c-f383-41a8-8f16-6ebd69d3b836 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:50.098200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 718/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:51.254722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6423a-a5df-4d98-8843-ca19a8540758 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:51.255255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6423a-a5df-4d98-8843-ca19a8540758 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:51.255255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6423a-a5df-4d98-8843-ca19a8540758 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:51.256594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:51.263333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf61ed3-2315-429c-aa25-06ca1591dbd3 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:51.263765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf61ed3-2315-429c-aa25-06ca1591dbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:51.263859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf61ed3-2315-429c-aa25-06ca1591dbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:51.317449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:51.339261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2bf61ed3-2315-429c-aa25-06ca1591dbd3 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:51.340458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2bf61ed3-2315-429c-aa25-06ca1591dbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:51.437086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:51.587363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6423a-a5df-4d98-8843-ca19a8540758 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:51.589748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-abc6423a-a5df-4d98-8843-ca19a8540758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:51.734527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6423a-a5df-4d98-8843-ca19a8540758 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:51.746649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:51.847278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86810a-1d83-4ba9-ba85-77d01eb39215 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:51.847811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86810a-1d83-4ba9-ba85-77d01eb39215 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:51.848089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86810a-1d83-4ba9-ba85-77d01eb39215 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:51.848383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86810a-1d83-4ba9-ba85-77d01eb39215 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:51.849972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0f86810a-1d83-4ba9-ba85-77d01eb39215 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:51.937185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86810a-1d83-4ba9-ba85-77d01eb39215 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:51.938775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86810a-1d83-4ba9-ba85-77d01eb39215 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:51.956500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 703/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:53.095898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dde3be-951e-43f0-b5e3-2c0b750bf201 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:53.096463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dde3be-951e-43f0-b5e3-2c0b750bf201 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:53.096463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dde3be-951e-43f0-b5e3-2c0b750bf201 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:53.097756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:53.104615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd9dd9-577d-44be-af05-c605184bbffe None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:53.105077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd9dd9-577d-44be-af05-c605184bbffe None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:53.105193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd9dd9-577d-44be-af05-c605184bbffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:53.124595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:53.180488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0dcd9dd9-577d-44be-af05-c605184bbffe None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:53.181942 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0dcd9dd9-577d-44be-af05-c605184bbffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:53.279878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:53.436638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dde3be-951e-43f0-b5e3-2c0b750bf201 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:53.438230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-07dde3be-951e-43f0-b5e3-2c0b750bf201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:53.549322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:53.585985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dde3be-951e-43f0-b5e3-2c0b750bf201 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:53.699492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75228981-3cbc-4686-877e-0fc6ff010bde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:53.700030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228981-3cbc-4686-877e-0fc6ff010bde service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:53.700307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228981-3cbc-4686-877e-0fc6ff010bde service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:53.700577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75228981-3cbc-4686-877e-0fc6ff010bde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:53.702154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-75228981-3cbc-4686-877e-0fc6ff010bde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:53.780199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:53.794019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75228981-3cbc-4686-877e-0fc6ff010bde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:53.795630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75228981-3cbc-4686-877e-0fc6ff010bde service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:53.813902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 721/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:54.938838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffdb98-95b9-4c9e-940a-a56cec52778c None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:54.939416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffdb98-95b9-4c9e-940a-a56cec52778c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:54.939416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffdb98-95b9-4c9e-940a-a56cec52778c None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:54.940772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:54.947631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532223d1-9243-40c0-9d60-e7756b971710 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:54.948080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532223d1-9243-40c0-9d60-e7756b971710 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:54.948229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532223d1-9243-40c0-9d60-e7756b971710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:55.024082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-532223d1-9243-40c0-9d60-e7756b971710 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:55.025296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-532223d1-9243-40c0-9d60-e7756b971710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:55.120170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:55.268256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffdb98-95b9-4c9e-940a-a56cec52778c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:55.269691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9dffdb98-95b9-4c9e-940a-a56cec52778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:55.352341 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:55.418850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffdb98-95b9-4c9e-940a-a56cec52778c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:55.532306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aef63-f4d3-45e5-b988-31f4352fc259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:55.532825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aef63-f4d3-45e5-b988-31f4352fc259 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:55.533103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aef63-f4d3-45e5-b988-31f4352fc259 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:55.533374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aef63-f4d3-45e5-b988-31f4352fc259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:55.535007 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5c4aef63-f4d3-45e5-b988-31f4352fc259 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:55.582896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:55.624111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4aef63-f4d3-45e5-b988-31f4352fc259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:55.625720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4aef63-f4d3-45e5-b988-31f4352fc259 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:55.644146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 706/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:56.798213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dffaa-ecc9-43c9-8f77-cb725fb63bc2 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:56.798748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dffaa-ecc9-43c9-8f77-cb725fb63bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:56.798748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153dffaa-ecc9-43c9-8f77-cb725fb63bc2 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:56.800065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:56.806493 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19771c91-cc02-4bb5-b993-453fb5439250 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:56.806916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19771c91-cc02-4bb5-b993-453fb5439250 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:56.807043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19771c91-cc02-4bb5-b993-453fb5439250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:56.881770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-19771c91-cc02-4bb5-b993-453fb5439250 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:56.882992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-19771c91-cc02-4bb5-b993-453fb5439250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:56.976804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:57.128512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dffaa-ecc9-43c9-8f77-cb725fb63bc2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:57.130037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-153dffaa-ecc9-43c9-8f77-cb725fb63bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:57.179092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:57.276968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153dffaa-ecc9-43c9-8f77-cb725fb63bc2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:57.388138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7d5bb-ea9e-403c-b0e9-f0451f6bf773 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:57.388689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d5bb-ea9e-403c-b0e9-f0451f6bf773 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:57.388944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d5bb-ea9e-403c-b0e9-f0451f6bf773 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:57.389211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d5bb-ea9e-403c-b0e9-f0451f6bf773 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:57.390855 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c3a7d5bb-ea9e-403c-b0e9-f0451f6bf773 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:57.397660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:57.482795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7d5bb-ea9e-403c-b0e9-f0451f6bf773 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:57.484400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7d5bb-ea9e-403c-b0e9-f0451f6bf773 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:57.502470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 724/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:14:58.597210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ec682-3178-4a09-a599-45cfc6ee7cd5 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:58.597776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ec682-3178-4a09-a599-45cfc6ee7cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:58.597776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ec682-3178-4a09-a599-45cfc6ee7cd5 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:58.599127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:14:58.610684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918def85-276f-4d61-b0b7-9d077ca5b8ef None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:58.611183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918def85-276f-4d61-b0b7-9d077ca5b8ef None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:58.611292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918def85-276f-4d61-b0b7-9d077ca5b8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:58.688033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-918def85-276f-4d61-b0b7-9d077ca5b8ef None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:14:58.689155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-918def85-276f-4d61-b0b7-9d077ca5b8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.760827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:58.789632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:14:58.942049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ec682-3178-4a09-a599-45cfc6ee7cd5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:14:58.943559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ab9ec682-3178-4a09-a599-45cfc6ee7cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:58.961335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:59.098369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ec682-3178-4a09-a599-45cfc6ee7cd5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:14:59.186177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:14:59.211554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e616ea-d05b-4a2c-a04a-8b915617b03e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:14:59.212095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e616ea-d05b-4a2c-a04a-8b915617b03e service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:14:59.212368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e616ea-d05b-4a2c-a04a-8b915617b03e service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:14:59.212714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e616ea-d05b-4a2c-a04a-8b915617b03e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:14:59.214586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-02e616ea-d05b-4a2c-a04a-8b915617b03e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:14:59.303370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e616ea-d05b-4a2c-a04a-8b915617b03e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:14:59.304943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e616ea-d05b-4a2c-a04a-8b915617b03e service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:14:59.323684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 709/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:14:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:00.459518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5994f-f0c6-4498-8532-3877f8cfc75a None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.459999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5994f-f0c6-4498-8532-3877f8cfc75a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.460117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5994f-f0c6-4498-8532-3877f8cfc75a None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.461404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:00.468554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e8d6-9363-44d7-9e03-3939a5a62788 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:00.469267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e8d6-9363-44d7-9e03-3939a5a62788 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:00.469364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e8d6-9363-44d7-9e03-3939a5a62788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:00.545500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-acc4e8d6-9363-44d7-9e03-3939a5a62788 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:00.546753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-acc4e8d6-9363-44d7-9e03-3939a5a62788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:00.587309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:00.642522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:00.801040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5994f-f0c6-4498-8532-3877f8cfc75a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:00.802515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-24a5994f-f0c6-4498-8532-3877f8cfc75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:00.946319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a5994f-f0c6-4498-8532-3877f8cfc75a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:01.001594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:01.059363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c539a0-256c-4396-87f7-513be3aa7b5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:01.059888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c539a0-256c-4396-87f7-513be3aa7b5b service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:01.060163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c539a0-256c-4396-87f7-513be3aa7b5b service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:01.060430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c539a0-256c-4396-87f7-513be3aa7b5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:01.062014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-73c539a0-256c-4396-87f7-513be3aa7b5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:01.152605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c539a0-256c-4396-87f7-513be3aa7b5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:01.154179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c539a0-256c-4396-87f7-513be3aa7b5b service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:01.172517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 727/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:02.307645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e62fa-1362-4736-b70f-28aa1b4e71de None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:02.308205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e62fa-1362-4736-b70f-28aa1b4e71de None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:02.308205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e62fa-1362-4736-b70f-28aa1b4e71de None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:02.309615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:15:02.315866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0421421-6a87-4f12-b1b7-c07774a8862c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:02.316319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0421421-6a87-4f12-b1b7-c07774a8862c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:02.316431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0421421-6a87-4f12-b1b7-c07774a8862c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:02.393455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c0421421-6a87-4f12-b1b7-c07774a8862c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:02.394706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c0421421-6a87-4f12-b1b7-c07774a8862c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:02.396685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:02.494757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:02.644995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e62fa-1362-4736-b70f-28aa1b4e71de None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:02.646541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3b5e62fa-1362-4736-b70f-28aa1b4e71de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:02.800942 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e62fa-1362-4736-b70f-28aa1b4e71de None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:02.828357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:02.916445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846142f-72fb-4763-b14e-7d503ae3072e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:02.916927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846142f-72fb-4763-b14e-7d503ae3072e service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:02.917231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846142f-72fb-4763-b14e-7d503ae3072e service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:02.917504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846142f-72fb-4763-b14e-7d503ae3072e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:02.919188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9846142f-72fb-4763-b14e-7d503ae3072e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:03.009209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846142f-72fb-4763-b14e-7d503ae3072e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:03.011423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846142f-72fb-4763-b14e-7d503ae3072e service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:03.030323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 712/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:05.180797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd66f3-197c-4edd-8729-6499e2cd671d None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:05.181935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd66f3-197c-4edd-8729-6499e2cd671d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:05.182066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd66f3-197c-4edd-8729-6499e2cd671d None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:05.183471 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:05.188968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebcf41-0ffc-4984-be2b-2679a6aac9ca None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:05.189395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebcf41-0ffc-4984-be2b-2679a6aac9ca None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:05.189501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebcf41-0ffc-4984-be2b-2679a6aac9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:05.231180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:05.255997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a0ebcf41-0ffc-4984-be2b-2679a6aac9ca None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:05.257217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a0ebcf41-0ffc-4984-be2b-2679a6aac9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:05.352981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:05.637239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd66f3-197c-4edd-8729-6499e2cd671d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:05.638845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-87bd66f3-197c-4edd-8729-6499e2cd671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:05.741548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd66f3-197c-4edd-8729-6499e2cd671d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:05.774457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:05.814718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18c278-1c6d-4815-b7c0-7b0f0b6e8664 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:05.815316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c278-1c6d-4815-b7c0-7b0f0b6e8664 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:05.815588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c278-1c6d-4815-b7c0-7b0f0b6e8664 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:05.815854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c278-1c6d-4815-b7c0-7b0f0b6e8664 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:05.817527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6d18c278-1c6d-4815-b7c0-7b0f0b6e8664 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:05.926004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18c278-1c6d-4815-b7c0-7b0f0b6e8664 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:05.927376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18c278-1c6d-4815-b7c0-7b0f0b6e8664 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:05.945413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 730/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:08.228751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81570d22-c1ae-4056-82a6-2c1d11b3f233 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:08.229330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81570d22-c1ae-4056-82a6-2c1d11b3f233 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:08.229330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81570d22-c1ae-4056-82a6-2c1d11b3f233 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:08.230687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:08.235771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0774a0c-5682-4443-a99c-6ea6f5af7431 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:08.236191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0774a0c-5682-4443-a99c-6ea6f5af7431 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:08.236298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0774a0c-5682-4443-a99c-6ea6f5af7431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:08.301803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d0774a0c-5682-4443-a99c-6ea6f5af7431 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:08.303115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d0774a0c-5682-4443-a99c-6ea6f5af7431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:08.318313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:08.404407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:08.695163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81570d22-c1ae-4056-82a6-2c1d11b3f233 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:08.696731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-81570d22-c1ae-4056-82a6-2c1d11b3f233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:08.800170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81570d22-c1ae-4056-82a6-2c1d11b3f233 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:08.813105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:08.873686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a7dc8-2c52-4153-be07-57e33d416118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:08.874277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a7dc8-2c52-4153-be07-57e33d416118 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:08.874576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a7dc8-2c52-4153-be07-57e33d416118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:08.874845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a7dc8-2c52-4153-be07-57e33d416118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:08.876423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1a7a7dc8-2c52-4153-be07-57e33d416118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:08.983939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a7dc8-2c52-4153-be07-57e33d416118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:08.985291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a7dc8-2c52-4153-be07-57e33d416118 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:09.003390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 715/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:11.525798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f75d5-5b2f-42e2-a3c1-250ff0342164 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:11.526391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f75d5-5b2f-42e2-a3c1-250ff0342164 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:11.526515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f75d5-5b2f-42e2-a3c1-250ff0342164 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:11.527784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:11.533291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b38b7-0e9f-4f85-858d-f32993bc79bb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:11.533716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b38b7-0e9f-4f85-858d-f32993bc79bb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:11.533811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b38b7-0e9f-4f85-858d-f32993bc79bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:11.583068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:11.600908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ce3b38b7-0e9f-4f85-858d-f32993bc79bb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:11.602129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ce3b38b7-0e9f-4f85-858d-f32993bc79bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:11.697677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:11.982855 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f75d5-5b2f-42e2-a3c1-250ff0342164 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:11.984427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-511f75d5-5b2f-42e2-a3c1-250ff0342164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:12.088910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f75d5-5b2f-42e2-a3c1-250ff0342164 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:12.129646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:12.162694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f905b2-7837-423c-a38f-c42e933225b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:12.163325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f905b2-7837-423c-a38f-c42e933225b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:12.163598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f905b2-7837-423c-a38f-c42e933225b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:12.163867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f905b2-7837-423c-a38f-c42e933225b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:12.165945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a6f905b2-7837-423c-a38f-c42e933225b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:12.280077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f905b2-7837-423c-a38f-c42e933225b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:12.281413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f905b2-7837-423c-a38f-c42e933225b0 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:12.300086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 733/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:14.683694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31253b9e-d0d3-4eab-bac2-1daf0d6d9011 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:14.684218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31253b9e-d0d3-4eab-bac2-1daf0d6d9011 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:14.684218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31253b9e-d0d3-4eab-bac2-1daf0d6d9011 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:14.685592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:14.691442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f18a3-99d2-473f-8d5b-62042733ccde None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:14.691922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f18a3-99d2-473f-8d5b-62042733ccde None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:14.692057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f18a3-99d2-473f-8d5b-62042733ccde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:14.758024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a88f18a3-99d2-473f-8d5b-62042733ccde None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:14.759302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a88f18a3-99d2-473f-8d5b-62042733ccde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:14.780141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:14.857602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:15.142951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31253b9e-d0d3-4eab-bac2-1daf0d6d9011 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:15.144494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-31253b9e-d0d3-4eab-bac2-1daf0d6d9011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:15.248480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31253b9e-d0d3-4eab-bac2-1daf0d6d9011 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:15.268998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:15.321124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929ad6a-8074-4385-8536-79d8b69dd1cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:15.321804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929ad6a-8074-4385-8536-79d8b69dd1cb service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:15.322080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929ad6a-8074-4385-8536-79d8b69dd1cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:15.322385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929ad6a-8074-4385-8536-79d8b69dd1cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:15.323967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c929ad6a-8074-4385-8536-79d8b69dd1cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:15.434482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929ad6a-8074-4385-8536-79d8b69dd1cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:15.435915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929ad6a-8074-4385-8536-79d8b69dd1cb service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:15.453770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 718/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:17.656000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfa907-a93f-4d37-aabd-40b3d39b7541 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:17.656529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfa907-a93f-4d37-aabd-40b3d39b7541 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:17.656529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cfa907-a93f-4d37-aabd-40b3d39b7541 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:17.657840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:17.663278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a86a-e58e-4120-99ea-60d7063df036 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:17.663703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a86a-e58e-4120-99ea-60d7063df036 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:17.663829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a86a-e58e-4120-99ea-60d7063df036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:17.728928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:17.731774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5801a86a-e58e-4120-99ea-60d7063df036 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:17.733001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5801a86a-e58e-4120-99ea-60d7063df036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:17.827990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:18.119754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfa907-a93f-4d37-aabd-40b3d39b7541 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:18.121355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-33cfa907-a93f-4d37-aabd-40b3d39b7541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:18.229276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cfa907-a93f-4d37-aabd-40b3d39b7541 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:18.278482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:18.306453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1dbd8-740a-4112-b119-c5524d7d3d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:18.307043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1dbd8-740a-4112-b119-c5524d7d3d74 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:18.307310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1dbd8-740a-4112-b119-c5524d7d3d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:18.307572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1dbd8-740a-4112-b119-c5524d7d3d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:18.309147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-66b1dbd8-740a-4112-b119-c5524d7d3d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:18.416434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1dbd8-740a-4112-b119-c5524d7d3d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:18.417781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1dbd8-740a-4112-b119-c5524d7d3d74 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:18.436448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 736/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:15:20.672720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d8fa-d4ce-4943-85e0-0758378662f9 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:20.673228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d8fa-d4ce-4943-85e0-0758378662f9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:20.673228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815d8fa-d4ce-4943-85e0-0758378662f9 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:20.674647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:20.680141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228333-65de-401e-8e54-996c65fba830 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:20.680552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228333-65de-401e-8e54-996c65fba830 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:20.680677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228333-65de-401e-8e54-996c65fba830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:20.747377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e0228333-65de-401e-8e54-996c65fba830 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:20.748602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e0228333-65de-401e-8e54-996c65fba830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:20.778386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:20.848502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:21.141533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815d8fa-d4ce-4943-85e0-0758378662f9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:21.143128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6815d8fa-d4ce-4943-85e0-0758378662f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:21.247054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815d8fa-d4ce-4943-85e0-0758378662f9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:21.274583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:21.320349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e3444-bb6f-4b4c-864f-82b717ae3945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:21.320948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e3444-bb6f-4b4c-864f-82b717ae3945 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:21.321224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e3444-bb6f-4b4c-864f-82b717ae3945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:21.321490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e3444-bb6f-4b4c-864f-82b717ae3945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:21.323446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5f0e3444-bb6f-4b4c-864f-82b717ae3945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:21.439015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e3444-bb6f-4b4c-864f-82b717ae3945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:21.440475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e3444-bb6f-4b4c-864f-82b717ae3945 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:21.458357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 721/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:23.783247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654790f-e423-42f6-a51b-1587cc500f7a None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:23.784263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654790f-e423-42f6-a51b-1587cc500f7a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:23.784263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654790f-e423-42f6-a51b-1587cc500f7a None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:23.785679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:23.792422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf40fcd-c6f5-44c6-83ea-3fde9dd6579c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:23.792892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf40fcd-c6f5-44c6-83ea-3fde9dd6579c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:23.792998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf40fcd-c6f5-44c6-83ea-3fde9dd6579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:23.872863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-abf40fcd-c6f5-44c6-83ea-3fde9dd6579c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:23.874136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-abf40fcd-c6f5-44c6-83ea-3fde9dd6579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:23.883965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:23.980644 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:24.297967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654790f-e423-42f6-a51b-1587cc500f7a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:24.299531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9654790f-e423-42f6-a51b-1587cc500f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:24.402982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654790f-e423-42f6-a51b-1587cc500f7a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:24.458517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:24.476599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048521d-7b12-4c6c-a53a-8051523026da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:24.477510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048521d-7b12-4c6c-a53a-8051523026da service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:24.477510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048521d-7b12-4c6c-a53a-8051523026da service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:24.477823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048521d-7b12-4c6c-a53a-8051523026da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:24.479803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a048521d-7b12-4c6c-a53a-8051523026da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:24.590226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048521d-7b12-4c6c-a53a-8051523026da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:24.591837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048521d-7b12-4c6c-a53a-8051523026da service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:24.611049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 739/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:26.827936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109831-c1f3-43fa-b367-2dca4d446b42 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:26.828491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109831-c1f3-43fa-b367-2dca4d446b42 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:26.828491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109831-c1f3-43fa-b367-2dca4d446b42 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:26.829836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:26.835350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd25da-6a5a-4aa2-97a8-1fc4fcf376b3 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:26.835765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd25da-6a5a-4aa2-97a8-1fc4fcf376b3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:26.835883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd25da-6a5a-4aa2-97a8-1fc4fcf376b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:26.902013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-05dd25da-6a5a-4aa2-97a8-1fc4fcf376b3 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:26.903249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-05dd25da-6a5a-4aa2-97a8-1fc4fcf376b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:26.940122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:26.996939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:27.278587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109831-c1f3-43fa-b367-2dca4d446b42 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:27.280259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-65109831-c1f3-43fa-b367-2dca4d446b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:27.383036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109831-c1f3-43fa-b367-2dca4d446b42 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:27.421155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:27.461221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d73eb-7d1c-4c2e-bc33-471f8dce9a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:27.461824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d73eb-7d1c-4c2e-bc33-471f8dce9a9b service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:27.462099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d73eb-7d1c-4c2e-bc33-471f8dce9a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:27.462673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d73eb-7d1c-4c2e-bc33-471f8dce9a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:27.464561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-288d73eb-7d1c-4c2e-bc33-471f8dce9a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:27.570722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d73eb-7d1c-4c2e-bc33-471f8dce9a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:27.572800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d73eb-7d1c-4c2e-bc33-471f8dce9a9b service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:27.590828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 724/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:29.867591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ffbe6-ce60-42ea-bdef-f65134c455e3 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:29.868165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ffbe6-ce60-42ea-bdef-f65134c455e3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:29.868293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ffbe6-ce60-42ea-bdef-f65134c455e3 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:29.870215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:29.875544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49142579-dddb-403d-8d18-fd74f1a8def8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:29.875956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49142579-dddb-403d-8d18-fd74f1a8def8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:29.876096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49142579-dddb-403d-8d18-fd74f1a8def8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:29.941285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-49142579-dddb-403d-8d18-fd74f1a8def8 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:29.942535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-49142579-dddb-403d-8d18-fd74f1a8def8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:29.960259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:30.039097 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:30.325572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ffbe6-ce60-42ea-bdef-f65134c455e3 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:30.327147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1b9ffbe6-ce60-42ea-bdef-f65134c455e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:30.436049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ffbe6-ce60-42ea-bdef-f65134c455e3 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:30.504043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:30.510593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfee0ed-0cb6-4394-accd-8acc1f7e8cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:30.511189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee0ed-0cb6-4394-accd-8acc1f7e8cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:30.511460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee0ed-0cb6-4394-accd-8acc1f7e8cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:30.511722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfee0ed-0cb6-4394-accd-8acc1f7e8cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:30.513287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0bfee0ed-0cb6-4394-accd-8acc1f7e8cb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:30.621245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfee0ed-0cb6-4394-accd-8acc1f7e8cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:30.622611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfee0ed-0cb6-4394-accd-8acc1f7e8cb4 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:30.641448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 742/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:32.802204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262692-6ac7-479c-8220-656ff539fd52 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:32.802772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262692-6ac7-479c-8220-656ff539fd52 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:32.802772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262692-6ac7-479c-8220-656ff539fd52 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:32.804085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:32.810235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a2426-9d05-4fa2-adfb-8ce9de29fb64 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:32.810675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a2426-9d05-4fa2-adfb-8ce9de29fb64 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:32.810805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a2426-9d05-4fa2-adfb-8ce9de29fb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:32.878440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7b9a2426-9d05-4fa2-adfb-8ce9de29fb64 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:32.879665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7b9a2426-9d05-4fa2-adfb-8ce9de29fb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:32.927424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:32.975874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:33.262871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262692-6ac7-479c-8220-656ff539fd52 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:33.264479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-42262692-6ac7-479c-8220-656ff539fd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:33.367908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262692-6ac7-479c-8220-656ff539fd52 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:33.414949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:33.443719 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10acc2-d8aa-4730-aaa9-33a2425e3f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:33.444318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10acc2-d8aa-4730-aaa9-33a2425e3f55 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:33.444598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10acc2-d8aa-4730-aaa9-33a2425e3f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:33.445116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10acc2-d8aa-4730-aaa9-33a2425e3f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:33.447794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bd10acc2-d8aa-4730-aaa9-33a2425e3f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:33.556361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10acc2-d8aa-4730-aaa9-33a2425e3f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:33.557889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10acc2-d8aa-4730-aaa9-33a2425e3f55 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:33.576800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 727/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:42.974432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=22 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:43.574528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=66 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:54.055219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35886b92-0544-419d-87f0-b370eac3a26e None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:54.055908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35886b92-0544-419d-87f0-b370eac3a26e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:54.055908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35886b92-0544-419d-87f0-b370eac3a26e None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:54.057715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 744/1471] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:54.062847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068b98f-378b-4232-b7e8-fa4cd2021307 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:54.063285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068b98f-378b-4232-b7e8-fa4cd2021307 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:54.063389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068b98f-378b-4232-b7e8-fa4cd2021307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:54.128214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8068b98f-378b-4232-b7e8-fa4cd2021307 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:54.129433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8068b98f-378b-4232-b7e8-fa4cd2021307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:54.225929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 728/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:54.525897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ba85-3d0a-4d5e-bdfb-930b828c875c None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:54.526168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ba85-3d0a-4d5e-bdfb-930b828c875c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:54.526468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ba85-3d0a-4d5e-bdfb-930b828c875c None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:54.527785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 745/1473] 10.222.0.20 () {54 vars in 966 bytes} [Fri Sep 9 23:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:54.532594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cc21e-0bbc-4f2f-90b9-a908ab1e3400 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:54.532873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cc21e-0bbc-4f2f-90b9-a908ab1e3400 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:54.533141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cc21e-0bbc-4f2f-90b9-a908ab1e3400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:54.610161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-308cc21e-0bbc-4f2f-90b9-a908ab1e3400 None None] MFA Rules not processed for user `c1842c116caa4a5d9d75c8605eeec0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:54.611392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-308cc21e-0bbc-4f2f-90b9-a908ab1e3400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:54.631359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:54.687221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 729/1474] 10.222.0.20 () {58 vars in 1063 bytes} [Fri Sep 9 23:15:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:54.694881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1ba85-3d0a-4d5e-bdfb-930b828c875c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:54.697918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f8b1ba85-3d0a-4d5e-bdfb-930b828c875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:54.800675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1ba85-3d0a-4d5e-bdfb-930b828c875c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:54.874284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c6610-9621-44a6-aaa6-6832f62e3ce8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1842c116caa4a5d9d75c8605eeec0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:54.874921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6610-9621-44a6-aaa6-6832f62e3ce8 service glance] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:54.875199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6610-9621-44a6-aaa6-6832f62e3ce8 service glance] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:54.875469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c6610-9621-44a6-aaa6-6832f62e3ce8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:54.877062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-047c6610-9621-44a6-aaa6-6832f62e3ce8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:54.930487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:54.989059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c6610-9621-44a6-aaa6-6832f62e3ce8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:54.990566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c6610-9621-44a6-aaa6-6832f62e3ce8 service glance] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:55.010408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 746/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:15:57.258129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6c5e3-995a-4f55-8719-c7f9660921ee None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:57.258776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6c5e3-995a-4f55-8719-c7f9660921ee None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:57.258776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6c5e3-995a-4f55-8719-c7f9660921ee None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:57.260178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 730/1476] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:15:57.265516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716bc74-1f86-4a4d-b9d8-6634ebb6fd81 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:57.265941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716bc74-1f86-4a4d-b9d8-6634ebb6fd81 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:57.266114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716bc74-1f86-4a4d-b9d8-6634ebb6fd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:57.332513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7716bc74-1f86-4a4d-b9d8-6634ebb6fd81 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:15:57.333720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7716bc74-1f86-4a4d-b9d8-6634ebb6fd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:57.429124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 747/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:15:57.707101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6c5e3-995a-4f55-8719-c7f9660921ee None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:15:57.708657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-eae6c5e3-995a-4f55-8719-c7f9660921ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:15:57.838550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6c5e3-995a-4f55-8719-c7f9660921ee None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:15:57.887010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:15:57.929873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0dbaf-b906-4401-9c85-93c7a33a9e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:15:57.930496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0dbaf-b906-4401-9c85-93c7a33a9e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:15:57.930800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0dbaf-b906-4401-9c85-93c7a33a9e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:15:57.931217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0dbaf-b906-4401-9c85-93c7a33a9e46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:15:57.931846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0dbaf-b906-4401-9c85-93c7a33a9e46 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:15:57.933541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-a9e0dbaf-b906-4401-9c85-93c7a33a9e46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:15:57.933541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Sep 09 23:15:57.933541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: Sep 09 23:15:57.934702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0dbaf-b906-4401-9c85-93c7a33a9e46 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:15:57.953816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 731/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Fri Sep 9 23:15:57 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:16:00.145437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb48d7-57fc-4972-966d-8f50cd54046f None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:00.146271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb48d7-57fc-4972-966d-8f50cd54046f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:00.146271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb48d7-57fc-4972-966d-8f50cd54046f None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:00.147600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 748/1479] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:16:00.155086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bf9f-9609-4f16-a0ed-ac6003c4bd35 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:00.155515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bf9f-9609-4f16-a0ed-ac6003c4bd35 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:00.155613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8621bf9f-9609-4f16-a0ed-ac6003c4bd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:00.221465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8621bf9f-9609-4f16-a0ed-ac6003c4bd35 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:16:00.222716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8621bf9f-9609-4f16-a0ed-ac6003c4bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:00.317536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 732/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:16:00.616625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb48d7-57fc-4972-966d-8f50cd54046f None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:16:00.618148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-85fb48d7-57fc-4972-966d-8f50cd54046f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:00.627921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:00.765654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb48d7-57fc-4972-966d-8f50cd54046f None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:16:00.850111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:00.889690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9a94b-306c-4123-bc5e-57e84aaab34e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:16:00.890290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9a94b-306c-4123-bc5e-57e84aaab34e service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:00.890594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9a94b-306c-4123-bc5e-57e84aaab34e service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:00.890859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9a94b-306c-4123-bc5e-57e84aaab34e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:00.892435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-39e9a94b-306c-4123-bc5e-57e84aaab34e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:00.981660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9a94b-306c-4123-bc5e-57e84aaab34e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:16:00.983349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9a94b-306c-4123-bc5e-57e84aaab34e service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:16:01.001849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 749/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:16:03.404889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0df1b-3ae4-4106-8e2a-f1933c944863 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:03.405475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0df1b-3ae4-4106-8e2a-f1933c944863 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:03.405475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0df1b-3ae4-4106-8e2a-f1933c944863 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:03.406901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 733/1482] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:16:03.412103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192f0f-2330-40b0-be76-21ac8a857fdd None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:03.412555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192f0f-2330-40b0-be76-21ac8a857fdd None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:03.412667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41192f0f-2330-40b0-be76-21ac8a857fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:03.480203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-41192f0f-2330-40b0-be76-21ac8a857fdd None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:16:03.481403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-41192f0f-2330-40b0-be76-21ac8a857fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:03.529007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:03.579357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 750/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:16:03.875362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0df1b-3ae4-4106-8e2a-f1933c944863 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:16:03.876947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-93b0df1b-3ae4-4106-8e2a-f1933c944863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:03.902733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:03.984029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0df1b-3ae4-4106-8e2a-f1933c944863 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:16:04.056050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2101e-4f6f-49b4-8da5-0390804ad860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:16:04.057274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2101e-4f6f-49b4-8da5-0390804ad860 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:04.057550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2101e-4f6f-49b4-8da5-0390804ad860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:04.057825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2101e-4f6f-49b4-8da5-0390804ad860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:04.059456 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-89f2101e-4f6f-49b4-8da5-0390804ad860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:04.133633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:04.168580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2101e-4f6f-49b4-8da5-0390804ad860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:16:04.169993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2101e-4f6f-49b4-8da5-0390804ad860 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:16:04.188086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 734/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:16:06.470259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e889-db10-49b7-ac15-1a1ed9939a53 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:06.471117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e889-db10-49b7-ac15-1a1ed9939a53 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:06.471117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e889-db10-49b7-ac15-1a1ed9939a53 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:06.472438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 751/1485] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:16:06.477777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecfdb8-31a4-4732-bcd4-a2695f81444b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:06.478846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecfdb8-31a4-4732-bcd4-a2695f81444b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:06.479042 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecfdb8-31a4-4732-bcd4-a2695f81444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:06.545465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8fecfdb8-31a4-4732-bcd4-a2695f81444b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:16:06.546709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8fecfdb8-31a4-4732-bcd4-a2695f81444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:06.643828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 735/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:16:06.937561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e889-db10-49b7-ac15-1a1ed9939a53 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:16:06.939194 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8454e889-db10-49b7-ac15-1a1ed9939a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:07.050159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e889-db10-49b7-ac15-1a1ed9939a53 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:16:07.122534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:07.124063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44d8ce-8616-40fa-b1d5-492aea10e784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:16:07.124657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44d8ce-8616-40fa-b1d5-492aea10e784 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:07.124934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44d8ce-8616-40fa-b1d5-492aea10e784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:07.125204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44d8ce-8616-40fa-b1d5-492aea10e784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:07.126812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2c44d8ce-8616-40fa-b1d5-492aea10e784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:07.236803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44d8ce-8616-40fa-b1d5-492aea10e784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:16:07.238130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44d8ce-8616-40fa-b1d5-492aea10e784 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:16:07.256486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 752/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:16:09.501916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf1c1-b12d-418b-8275-eb5a5918694e None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:09.502522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf1c1-b12d-418b-8275-eb5a5918694e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:09.502522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abf1c1-b12d-418b-8275-eb5a5918694e None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:09.503847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 736/1488] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:16:09.509138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a36bf2-c9c4-4d7c-b42f-af4c051cc115 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:09.509723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a36bf2-c9c4-4d7c-b42f-af4c051cc115 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:09.510303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a36bf2-c9c4-4d7c-b42f-af4c051cc115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:09.576441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c2a36bf2-c9c4-4d7c-b42f-af4c051cc115 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:16:09.577633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c2a36bf2-c9c4-4d7c-b42f-af4c051cc115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:09.631074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:09.674909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 753/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:16:09.964559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf1c1-b12d-418b-8275-eb5a5918694e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:16:09.966106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d5abf1c1-b12d-418b-8275-eb5a5918694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:10.002441 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:10.069211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abf1c1-b12d-418b-8275-eb5a5918694e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:16:10.141992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd7017-cafc-42cf-8cb2-e2de51707cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:16:10.142615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd7017-cafc-42cf-8cb2-e2de51707cda service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:10.142887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd7017-cafc-42cf-8cb2-e2de51707cda service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:10.143150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd7017-cafc-42cf-8cb2-e2de51707cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:10.144710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fdfd7017-cafc-42cf-8cb2-e2de51707cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:10.223480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:10.252539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd7017-cafc-42cf-8cb2-e2de51707cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:16:10.253971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd7017-cafc-42cf-8cb2-e2de51707cda service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:16:10.272873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 737/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:16:12.853880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e1125-ce9d-41a1-99ac-13f978a22476 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:12.854454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e1125-ce9d-41a1-99ac-13f978a22476 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:12.854454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e1125-ce9d-41a1-99ac-13f978a22476 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:12.855771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 754/1491] 10.222.0.20 () {54 vars in 927 bytes} [Fri Sep 9 23:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:16:12.861453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c716b9-316d-4366-b272-eaab48df2f49 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:12.861933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c716b9-316d-4366-b272-eaab48df2f49 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:12.862032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c716b9-316d-4366-b272-eaab48df2f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:12.927234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-65c716b9-316d-4366-b272-eaab48df2f49 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:16:12.928506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-65c716b9-316d-4366-b272-eaab48df2f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:13.022493 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 738/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Fri Sep 9 23:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:16:13.319456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e1125-ce9d-41a1-99ac-13f978a22476 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:16:13.321576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fd2e1125-ce9d-41a1-99ac-13f978a22476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:13.425686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e1125-ce9d-41a1-99ac-13f978a22476 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:16:13.501522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557786e-32dc-4f24-82c1-6d5971e0f04e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:16:13.502120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557786e-32dc-4f24-82c1-6d5971e0f04e service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:16:13.502429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557786e-32dc-4f24-82c1-6d5971e0f04e service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:16:13.506475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557786e-32dc-4f24-82c1-6d5971e0f04e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:16:13.506475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4557786e-32dc-4f24-82c1-6d5971e0f04e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:16:13.513419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:13.615989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557786e-32dc-4f24-82c1-6d5971e0f04e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:16:13.617407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557786e-32dc-4f24-82c1-6d5971e0f04e service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:16:13.636389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 755/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:16:23.021018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=82 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:16:23.633908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=50 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:39.337236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbdc1e-dc66-4de9-b949-0fcea1123ec5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:39.337849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbdc1e-dc66-4de9-b949-0fcea1123ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:39.337849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbdc1e-dc66-4de9-b949-0fcea1123ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:39.424881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d4bbdc1e-dc66-4de9-b949-0fcea1123ec5 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:17:39.427698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d4bbdc1e-dc66-4de9-b949-0fcea1123ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:39.520517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 739/1494] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:17:39.528390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f7204-8449-40ed-8f3f-987f675a4d10 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:39.528913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f7204-8449-40ed-8f3f-987f675a4d10 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:39.529010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f7204-8449-40ed-8f3f-987f675a4d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:39.606802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-434f7204-8449-40ed-8f3f-987f675a4d10 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:17:39.608213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-434f7204-8449-40ed-8f3f-987f675a4d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:39.704028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 756/1495] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:17:39.708709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbdc1e-dc66-4de9-b949-0fcea1123ec5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:39.709908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d4bbdc1e-dc66-4de9-b949-0fcea1123ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:39.722414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbdc1e-dc66-4de9-b949-0fcea1123ec5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:39.783303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:39.815969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:39.816628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:39.817094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:39.817553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:39.818523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:17:39.818821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:39.819831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:17:39.819831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Sep 09 23:17:39.819831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: Sep 09 23:17:39.820721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:39.832226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 740/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:17:39.840585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f7204-8449-40ed-8f3f-987f675a4d10 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:39.850417 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f7204-8449-40ed-8f3f-987f675a4d10 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:39.910922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:39.944428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:39.945028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:39.945310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:39.945579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:39.946223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:39.947964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WARNING py.warnings [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:17:39.947964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: warnings.warn(msg) Sep 09 23:17:39.947964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: Sep 09 23:17:39.949181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:40.063627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 757/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:17:39 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:17:40.067840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:40.069383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:40.103377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601994e6-fcae-40c2-b649-0922f564d1eb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:40.196038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:40.196641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:40.196915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:40.197179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:40.197794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:40.198813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:17:40.198813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Sep 09 23:17:40.198813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: Sep 09 23:17:40.200052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:40.298460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 741/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:17:40 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:17:40.302604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:40.304168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:40.365269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b942cf-b5b1-4d0e-a5e0-ed283f868f16 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:40.449306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:40.459727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:40.460382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:40.461452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:40.461452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:40.461690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:40.462743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WARNING py.warnings [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:17:40.462743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: warnings.warn(msg) Sep 09 23:17:40.462743 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: Sep 09 23:17:40.463621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:40.478257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 758/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:17:40.482950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:40.484537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:40.495996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea7bb8-6cec-4f95-aa43-a2867cf0a63e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:40.521322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:40.590880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6dfd2-b9f8-4c1b-ab91-65f21e53e032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:40.591571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6dfd2-b9f8-4c1b-ab91-65f21e53e032 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:40.591881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6dfd2-b9f8-4c1b-ab91-65f21e53e032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:40.592154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6dfd2-b9f8-4c1b-ab91-65f21e53e032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:40.592801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6dfd2-b9f8-4c1b-ab91-65f21e53e032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:40.593822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-f5b6dfd2-b9f8-4c1b-ab91-65f21e53e032 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:17:40.593822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Sep 09 23:17:40.593822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: Sep 09 23:17:40.594746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6dfd2-b9f8-4c1b-ab91-65f21e53e032 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:40.608520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 742/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:17:40.612786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:40.614297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:40.647689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2751-8e9a-467b-88ac-b5da271eab8d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:40.739145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bba6ad-049d-4dae-9459-4bc61326dfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:40.739712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bba6ad-049d-4dae-9459-4bc61326dfd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:40.739981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bba6ad-049d-4dae-9459-4bc61326dfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:40.740247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bba6ad-049d-4dae-9459-4bc61326dfd1 admin admin] PATH_INFO: `/v3/projects/23d641de89fa4e51b858b0e6f2df1ebf/users/0f722b5deef541598ee1eb0c20ba5678/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:40.762399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bba6ad-049d-4dae-9459-4bc61326dfd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d641de89fa4e51b858b0e6f2df1ebf, user_id=0f722b5deef541598ee1eb0c20ba5678, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:40.763565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WARNING py.warnings [None req-17bba6ad-049d-4dae-9459-4bc61326dfd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:17:40.763565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: warnings.warn(msg) Sep 09 23:17:40.763565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: Sep 09 23:17:40.765468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bba6ad-049d-4dae-9459-4bc61326dfd1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:40.840244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 759/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:17:40 2022] PUT /identity/v3/projects/23d641de89fa4e51b858b0e6f2df1ebf/users/0f722b5deef541598ee1eb0c20ba5678/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:17:40.853212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03647ba3-2a4a-42bb-b5b2-9d8c59572ec8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:40.853492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03647ba3-2a4a-42bb-b5b2-9d8c59572ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:40.853762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03647ba3-2a4a-42bb-b5b2-9d8c59572ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:40.909953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-03647ba3-2a4a-42bb-b5b2-9d8c59572ec8 None None] MFA Rules not processed for user `0f722b5deef541598ee1eb0c20ba5678`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:17:40.911174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-03647ba3-2a4a-42bb-b5b2-9d8c59572ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:40.987098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 743/1502] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:17:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:17:41.000487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be87c2-8f1e-4242-b4b5-f87d31ace663 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:41.000775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be87c2-8f1e-4242-b4b5-f87d31ace663 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:41.001048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be87c2-8f1e-4242-b4b5-f87d31ace663 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:41.002392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 760/1503] 10.222.0.20 () {54 vars in 964 bytes} [Fri Sep 9 23:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:17:41.007068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51352-6981-4cc6-b538-8632916bd3de None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:41.007352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51352-6981-4cc6-b538-8632916bd3de None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:41.007624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c51352-6981-4cc6-b538-8632916bd3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:41.046406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:41.092481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-66c51352-6981-4cc6-b538-8632916bd3de None None] MFA Rules not processed for user `c54fab93990146e0bb746abfd0843444`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:17:41.093765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-66c51352-6981-4cc6-b538-8632916bd3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:41.169109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 744/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Fri Sep 9 23:17:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:17:41.177023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be87c2-8f1e-4242-b4b5-f87d31ace663 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:41.178606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a1be87c2-8f1e-4242-b4b5-f87d31ace663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:41.229451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:41.283344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be87c2-8f1e-4242-b4b5-f87d31ace663 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:41.356858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:41.357454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:41.357731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:41.357999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:41.359634 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:41.463372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:41.464796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:41.468611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:41.482698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 761/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:17:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:17:41.531764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e212-6cc4-4341-ac22-0ccd970d84a8 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:41.532179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e212-6cc4-4341-ac22-0ccd970d84a8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:41.532314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4e212-6cc4-4341-ac22-0ccd970d84a8 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:41.533619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 745/1506] 10.222.0.20 () {52 vars in 830 bytes} [Fri Sep 9 23:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:17:41.537248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:41.538522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:41.636469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63625c4-a5e8-4ac4-8886-286be6124324 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:41.728277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:41.728887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:41.729157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:41.729420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] PATH_INFO: `/v3/users/0f722b5deef541598ee1eb0c20ba5678` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:41.742449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f722b5deef541598ee1eb0c20ba5678)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:41.743451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WARNING py.warnings [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:17:41.743451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: warnings.warn(msg) Sep 09 23:17:41.743451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: Sep 09 23:17:41.744644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:41.754537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:41.988639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f722b5deef541598ee1eb0c20ba5678'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:41.989074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f722b5deef541598ee1eb0c20ba5678'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:42.062971 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f722b5deef541598ee1eb0c20ba5678'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:42.087069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f722b5deef541598ee1eb0c20ba5678'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:42.101867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f722b5deef541598ee1eb0c20ba5678'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:42.108174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 762/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:17:41 2022] DELETE /identity/v3/users/0f722b5deef541598ee1eb0c20ba5678 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:17:42.112529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779b24-ac6e-45c9-aab6-f248b92d6624 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:42.112814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779b24-ac6e-45c9-aab6-f248b92d6624 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:42.113086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e779b24-ac6e-45c9-aab6-f248b92d6624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:42.171120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9e779b24-ac6e-45c9-aab6-f248b92d6624 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:17:42.172317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9e779b24-ac6e-45c9-aab6-f248b92d6624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:42.246763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:42.259495 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 746/1508] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:17:42.269690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:42.270966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:42.371976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811a256-51db-4011-a30b-3fcf3cff20b2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:42.429661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:42.449210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e33027-a7b2-4834-ae13-d9a9c5573100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:42.449799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33027-a7b2-4834-ae13-d9a9c5573100 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:42.450078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33027-a7b2-4834-ae13-d9a9c5573100 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:42.450365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e33027-a7b2-4834-ae13-d9a9c5573100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:42.452763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c8e33027-a7b2-4834-ae13-d9a9c5573100 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:42.563152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e33027-a7b2-4834-ae13-d9a9c5573100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:42.564526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e33027-a7b2-4834-ae13-d9a9c5573100 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:42.583051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 763/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:17:43.116368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e779b24-ac6e-45c9-aab6-f248b92d6624 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:17:43.117593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9e779b24-ac6e-45c9-aab6-f248b92d6624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:17:43.129301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e779b24-ac6e-45c9-aab6-f248b92d6624 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:17:43.229492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:17:43.230147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:17:43.230447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:17:43.230730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] PATH_INFO: `/v3/projects/23d641de89fa4e51b858b0e6f2df1ebf` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:17:43.241570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d641de89fa4e51b858b0e6f2df1ebf)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:17:43.242616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:17:43.242616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Sep 09 23:17:43.242616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: Sep 09 23:17:43.243867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:17:43.452565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d641de89fa4e51b858b0e6f2df1ebf', 'request_id': 'req-4ecfddee-3b27-4264-b80b-615bf3932f78'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:43.543094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d641de89fa4e51b858b0e6f2df1ebf', 'request_id': 'req-4ecfddee-3b27-4264-b80b-615bf3932f78'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:43.549789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d641de89fa4e51b858b0e6f2df1ebf', 'request_id': 'req-4ecfddee-3b27-4264-b80b-615bf3932f78'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:43.590527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] The token cache is being invalidate because project 23d641de89fa4e51b858b0e6f2df1ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:17:43.590825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:43.592665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4ecfddee-3b27-4264-b80b-615bf3932f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:17:43.594114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 747/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:17:43 2022] DELETE /identity/v3/projects/23d641de89fa4e51b858b0e6f2df1ebf => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:17:52.580761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=62 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:17:53.588311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:19:07.585057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48cd24-d067-4416-8309-ef9d2940e035 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:07.585657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48cd24-d067-4416-8309-ef9d2940e035 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:07.585657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48cd24-d067-4416-8309-ef9d2940e035 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:07.586982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:07.592922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171196-6322-4731-82a4-e3361eaaba9f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:07.593513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171196-6322-4731-82a4-e3361eaaba9f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:07.593513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171196-6322-4731-82a4-e3361eaaba9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:07.673304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-af171196-6322-4731-82a4-e3361eaaba9f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:07.674441 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-af171196-6322-4731-82a4-e3361eaaba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:07.777561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:07.931345 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48cd24-d067-4416-8309-ef9d2940e035 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:07.933434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cd48cd24-d067-4416-8309-ef9d2940e035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:08.056241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48cd24-d067-4416-8309-ef9d2940e035 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:08.061734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121de79-5c08-45ac-9e27-62a5c883f403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:08.062328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121de79-5c08-45ac-9e27-62a5c883f403 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:08.062629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121de79-5c08-45ac-9e27-62a5c883f403 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:08.062898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121de79-5c08-45ac-9e27-62a5c883f403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:08.070728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121de79-5c08-45ac-9e27-62a5c883f403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:08.072283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121de79-5c08-45ac-9e27-62a5c883f403 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:08.081957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 765/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:19:09.167591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4c3e-64b5-46e4-9b4c-ae1e39c6eba8 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:09.168158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4c3e-64b5-46e4-9b4c-ae1e39c6eba8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:09.168158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4c3e-64b5-46e4-9b4c-ae1e39c6eba8 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:09.169542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:09.175678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee648d51-7248-44c1-916e-060d524f8c74 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:09.176117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee648d51-7248-44c1-916e-060d524f8c74 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:09.176219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee648d51-7248-44c1-916e-060d524f8c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:09.196181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ee648d51-7248-44c1-916e-060d524f8c74 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:09.197387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ee648d51-7248-44c1-916e-060d524f8c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:09.205835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:09.358639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe4c3e-64b5-46e4-9b4c-ae1e39c6eba8 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:09.373939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe4c3e-64b5-46e4-9b4c-ae1e39c6eba8 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:09.379513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a2572f-56e9-49ea-9bd7-ff554c761d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:09.380107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2572f-56e9-49ea-9bd7-ff554c761d77 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:09.380379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2572f-56e9-49ea-9bd7-ff554c761d77 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:09.380645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2572f-56e9-49ea-9bd7-ff554c761d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:09.389398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2572f-56e9-49ea-9bd7-ff554c761d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:09.390996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a2572f-56e9-49ea-9bd7-ff554c761d77 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:09.401815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 750/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:19:09.830508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb859ab-6582-4393-8afa-7d687208528d None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:09.830796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb859ab-6582-4393-8afa-7d687208528d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:09.831070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb859ab-6582-4393-8afa-7d687208528d None None] PATH_INFO: `/v3` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:09.832348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 767/1517] 10.222.0.42 () {54 vars in 925 bytes} [Fri Sep 9 23:19:09 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:19:09.861410 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c51b-0e8f-43d6-940c-e35b4bcf8fe5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:09.861699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c51b-0e8f-43d6-940c-e35b4bcf8fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:09.861979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33c51b-0e8f-43d6-940c-e35b4bcf8fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:09.922964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-da33c51b-0e8f-43d6-940c-e35b4bcf8fe5 None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:09.924178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-da33c51b-0e8f-43d6-940c-e35b4bcf8fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:09.940681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:19:09.996255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 751/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Fri Sep 9 23:19:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:13.810764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb6b91-db72-47f1-bf14-4a2de3c79fec None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:13.811279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb6b91-db72-47f1-bf14-4a2de3c79fec None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:13.811279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb6b91-db72-47f1-bf14-4a2de3c79fec None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:13.812599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 768/1519] 10.222.0.20 () {54 vars in 961 bytes} [Fri Sep 9 23:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:13.818859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202d8f-7bd8-4df6-9423-13ed53e4ca70 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:13.819297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202d8f-7bd8-4df6-9423-13ed53e4ca70 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:13.819454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d202d8f-7bd8-4df6-9423-13ed53e4ca70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:13.839690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8d202d8f-7bd8-4df6-9423-13ed53e4ca70 None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:13.840903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8d202d8f-7bd8-4df6-9423-13ed53e4ca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:13.850385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 752/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Fri Sep 9 23:19:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:13.857885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb6b91-db72-47f1-bf14-4a2de3c79fec None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:13.871261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb6b91-db72-47f1-bf14-4a2de3c79fec None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:13.876814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9792eed-90c5-4b8c-bf19-64f94cae0ced service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:13.877386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9792eed-90c5-4b8c-bf19-64f94cae0ced service placement] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:13.877660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9792eed-90c5-4b8c-bf19-64f94cae0ced service placement] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:13.877926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9792eed-90c5-4b8c-bf19-64f94cae0ced service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:13.886204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9792eed-90c5-4b8c-bf19-64f94cae0ced service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:13.887783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9792eed-90c5-4b8c-bf19-64f94cae0ced service placement] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:13.898090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 769/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:19:20.917304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d309d-2a70-47bb-ab30-acf36437466b None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:20.917871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d309d-2a70-47bb-ab30-acf36437466b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:20.917871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d309d-2a70-47bb-ab30-acf36437466b None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:20.919283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:20.925470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98921dc-eddc-4a32-afa3-1047f5eb31c0 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:20.925910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98921dc-eddc-4a32-afa3-1047f5eb31c0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:20.926011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98921dc-eddc-4a32-afa3-1047f5eb31c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:20.945295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e98921dc-eddc-4a32-afa3-1047f5eb31c0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:20.946519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e98921dc-eddc-4a32-afa3-1047f5eb31c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:20.954820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:21.109004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d309d-2a70-47bb-ab30-acf36437466b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:21.123748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d309d-2a70-47bb-ab30-acf36437466b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:21.129143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ba0e8-ba45-48c5-8c36-ed0bf821996d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:21.129728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ba0e8-ba45-48c5-8c36-ed0bf821996d service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:21.130248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ba0e8-ba45-48c5-8c36-ed0bf821996d service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:21.130551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ba0e8-ba45-48c5-8c36-ed0bf821996d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:21.140268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ba0e8-ba45-48c5-8c36-ed0bf821996d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:21.145317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ba0e8-ba45-48c5-8c36-ed0bf821996d service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:21.151426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 754/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:19:22.230815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3b94-a143-4d62-828d-f2dc3eabb2ed None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:22.231364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3b94-a143-4d62-828d-f2dc3eabb2ed None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:22.231364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3b94-a143-4d62-828d-f2dc3eabb2ed None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:22.233301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 9 23:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:22.239506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ea8c8-1b84-4c87-b7aa-282b80f69a24 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:22.239945 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ea8c8-1b84-4c87-b7aa-282b80f69a24 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:22.240052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ea8c8-1b84-4c87-b7aa-282b80f69a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:22.259964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-354ea8c8-1b84-4c87-b7aa-282b80f69a24 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:22.261169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-354ea8c8-1b84-4c87-b7aa-282b80f69a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:22.269161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 9 23:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:22.422032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda3b94-a143-4d62-828d-f2dc3eabb2ed None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:22.436899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda3b94-a143-4d62-828d-f2dc3eabb2ed None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:22.442657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74e6fd-80b6-4b29-abf4-b53e4dd2bb43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:22.443251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e6fd-80b6-4b29-abf4-b53e4dd2bb43 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:22.443525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e6fd-80b6-4b29-abf4-b53e4dd2bb43 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:22.443794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74e6fd-80b6-4b29-abf4-b53e4dd2bb43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:22.452595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74e6fd-80b6-4b29-abf4-b53e4dd2bb43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:22.454227 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74e6fd-80b6-4b29-abf4-b53e4dd2bb43 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:22.463583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 772/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:19:26.621908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc83cfe-fcc9-4ed3-8ae7-f9adeaa0ea22 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:26.622549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc83cfe-fcc9-4ed3-8ae7-f9adeaa0ea22 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:26.622647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc83cfe-fcc9-4ed3-8ae7-f9adeaa0ea22 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:26.623935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:26.629193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa1c69-646a-488b-b9a0-51045de9bd5a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:26.629605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa1c69-646a-488b-b9a0-51045de9bd5a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:26.629745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa1c69-646a-488b-b9a0-51045de9bd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:26.649211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f1fa1c69-646a-488b-b9a0-51045de9bd5a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:26.650440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f1fa1c69-646a-488b-b9a0-51045de9bd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:26.658923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:26.946515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc83cfe-fcc9-4ed3-8ae7-f9adeaa0ea22 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:26.948200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cbc83cfe-fcc9-4ed3-8ae7-f9adeaa0ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:27.031705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc83cfe-fcc9-4ed3-8ae7-f9adeaa0ea22 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:27.035226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59061d16-50fc-4fd9-9870-a6b72197be63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:27.035817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59061d16-50fc-4fd9-9870-a6b72197be63 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:27.036094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59061d16-50fc-4fd9-9870-a6b72197be63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:27.036358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59061d16-50fc-4fd9-9870-a6b72197be63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:27.045244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59061d16-50fc-4fd9-9870-a6b72197be63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:27.046871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59061d16-50fc-4fd9-9870-a6b72197be63 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:27.056893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 757/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:19:29.216707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc30c66-4eaf-4746-9c2a-dbcc49472054 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:29.217277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc30c66-4eaf-4746-9c2a-dbcc49472054 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:29.217396 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc30c66-4eaf-4746-9c2a-dbcc49472054 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:29.218723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:19:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:29.224053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699e9dd-1623-441c-9b15-27691f54fd91 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:29.224507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699e9dd-1623-441c-9b15-27691f54fd91 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:29.224614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699e9dd-1623-441c-9b15-27691f54fd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:29.244877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3699e9dd-1623-441c-9b15-27691f54fd91 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:29.246077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3699e9dd-1623-441c-9b15-27691f54fd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:29.254654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:29.548286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc30c66-4eaf-4746-9c2a-dbcc49472054 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:29.550030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6bc30c66-4eaf-4746-9c2a-dbcc49472054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:29.562804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc30c66-4eaf-4746-9c2a-dbcc49472054 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:29.566423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc5dcf-29d6-4764-bbf0-a7464490c875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:29.567011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc5dcf-29d6-4764-bbf0-a7464490c875 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:29.567283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc5dcf-29d6-4764-bbf0-a7464490c875 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:29.567548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc5dcf-29d6-4764-bbf0-a7464490c875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:29.576386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc5dcf-29d6-4764-bbf0-a7464490c875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:29.577811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc5dcf-29d6-4764-bbf0-a7464490c875 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:29.587464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 775/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:19:33.239684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135787eb-39f5-4584-81d3-bccccd4677f7 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:33.240278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135787eb-39f5-4584-81d3-bccccd4677f7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:33.240278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135787eb-39f5-4584-81d3-bccccd4677f7 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:33.241584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:33.246940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56906f-1aa4-49f2-8589-e89bd520812e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:33.247444 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56906f-1aa4-49f2-8589-e89bd520812e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:33.247704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e56906f-1aa4-49f2-8589-e89bd520812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:33.255428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=81 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:19:33.267614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4e56906f-1aa4-49f2-8589-e89bd520812e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:33.268836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4e56906f-1aa4-49f2-8589-e89bd520812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:33.276891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:33.584010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135787eb-39f5-4584-81d3-bccccd4677f7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:33.585696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-135787eb-39f5-4584-81d3-bccccd4677f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:33.650358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:19:33.664203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135787eb-39f5-4584-81d3-bccccd4677f7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:33.667807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dd58a-2cc8-4d4b-8e6d-1c3b24a92a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:33.668403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dd58a-2cc8-4d4b-8e6d-1c3b24a92a93 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:33.668674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dd58a-2cc8-4d4b-8e6d-1c3b24a92a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:33.668996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dd58a-2cc8-4d4b-8e6d-1c3b24a92a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:33.679079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dd58a-2cc8-4d4b-8e6d-1c3b24a92a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:33.680516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dd58a-2cc8-4d4b-8e6d-1c3b24a92a93 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:33.689831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 760/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:19:35.855970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c902a-a32c-42d9-b7d7-f07642f08b83 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:35.857086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c902a-a32c-42d9-b7d7-f07642f08b83 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:35.857241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c902a-a32c-42d9-b7d7-f07642f08b83 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:35.858836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 9 23:19:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:19:35.864165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141183-513b-4f6d-bea6-a97c70ecc8f9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:35.864595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141183-513b-4f6d-bea6-a97c70ecc8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:35.864708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7141183-513b-4f6d-bea6-a97c70ecc8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:35.888957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c7141183-513b-4f6d-bea6-a97c70ecc8f9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:19:35.890169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c7141183-513b-4f6d-bea6-a97c70ecc8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:35.898089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 9 23:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:19:36.199912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c902a-a32c-42d9-b7d7-f07642f08b83 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:19:36.202278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9a7c902a-a32c-42d9-b7d7-f07642f08b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:19:36.215331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c902a-a32c-42d9-b7d7-f07642f08b83 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:19:36.218949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cae99-d16d-49d7-a45a-c7bb4180ff3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:19:36.219537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cae99-d16d-49d7-a45a-c7bb4180ff3f service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:19:36.219821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cae99-d16d-49d7-a45a-c7bb4180ff3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:19:36.220093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cae99-d16d-49d7-a45a-c7bb4180ff3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:19:36.229177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cae99-d16d-49d7-a45a-c7bb4180ff3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:19:36.230621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cae99-d16d-49d7-a45a-c7bb4180ff3f service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:19:36.240171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 778/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:19:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:19:45.883586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=11 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:19:46.235985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=8 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:02.234267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e3ce-3876-4479-bbe1-fa42e04099e9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.241120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e3ce-3876-4479-bbe1-fa42e04099e9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.241120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e3ce-3876-4479-bbe1-fa42e04099e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.263804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-92f2e3ce-3876-4479-bbe1-fa42e04099e9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:02.265447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-92f2e3ce-3876-4479-bbe1-fa42e04099e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:02.278406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 762/1540] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:02.288127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011968-bd4b-4b37-888b-9d2affe3ed5d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.289089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011968-bd4b-4b37-888b-9d2affe3ed5d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.289582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011968-bd4b-4b37-888b-9d2affe3ed5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.328293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-54011968-bd4b-4b37-888b-9d2affe3ed5d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:02.329898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-54011968-bd4b-4b37-888b-9d2affe3ed5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:02.343836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 779/1541] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:02.348855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e3ce-3876-4479-bbe1-fa42e04099e9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:02.363605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e3ce-3876-4479-bbe1-fa42e04099e9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:02.372863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:02.373801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.374412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.374917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.375797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:02.376884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:02.378927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:02.392477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 763/1542] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:02.401388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54011968-bd4b-4b37-888b-9d2affe3ed5d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:02.421815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54011968-bd4b-4b37-888b-9d2affe3ed5d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:02.431686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ae18-039f-491d-98e2-d65d2309fcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:02.432318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ae18-039f-491d-98e2-d65d2309fcce admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.432597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ae18-039f-491d-98e2-d65d2309fcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.432869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ae18-039f-491d-98e2-d65d2309fcce admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.433510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0ae18-039f-491d-98e2-d65d2309fcce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:02.435431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0ae18-039f-491d-98e2-d65d2309fcce admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:02.597722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 780/1543] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:02.601542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:02.617634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf4c49-193d-4949-a1a0-060b8cadef54 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:02.654432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230caef7-b9a7-4400-b922-590067fa7560 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.654772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230caef7-b9a7-4400-b922-590067fa7560 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.654961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230caef7-b9a7-4400-b922-590067fa7560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.688463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-230caef7-b9a7-4400-b922-590067fa7560 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:02.689863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-230caef7-b9a7-4400-b922-590067fa7560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:02.759122 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bbf68-ee3d-4037-b449-1d0f603fabe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:02.759794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bbf68-ee3d-4037-b449-1d0f603fabe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.760067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bbf68-ee3d-4037-b449-1d0f603fabe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.760332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bbf68-ee3d-4037-b449-1d0f603fabe3 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.760976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bbf68-ee3d-4037-b449-1d0f603fabe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:02.763163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bbf68-ee3d-4037-b449-1d0f603fabe3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:02.840700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 781/1544] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:02.841910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3c96-bb74-4cff-8b8d-471e1d0e8f16 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.842195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3c96-bb74-4cff-8b8d-471e1d0e8f16 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.842498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce3c96-bb74-4cff-8b8d-471e1d0e8f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.879523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-70ce3c96-bb74-4cff-8b8d-471e1d0e8f16 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:02.880676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-70ce3c96-bb74-4cff-8b8d-471e1d0e8f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:02.897955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 782/1545] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:02.899117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e043bf-1753-495c-9e6b-51a2c4ab67eb None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.905279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e043bf-1753-495c-9e6b-51a2c4ab67eb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.905279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e043bf-1753-495c-9e6b-51a2c4ab67eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.906063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 764/1546] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/users => generated 290 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:02.906063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb496aa-7089-42f2-92c9-9bbe015b9083 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.906063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb496aa-7089-42f2-92c9-9bbe015b9083 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.906063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb496aa-7089-42f2-92c9-9bbe015b9083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.932131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d4e043bf-1753-495c-9e6b-51a2c4ab67eb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:02.939757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d4e043bf-1753-495c-9e6b-51a2c4ab67eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:02.949587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0bb496aa-7089-42f2-92c9-9bbe015b9083 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:02.951510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 783/1547] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:02.951852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0bb496aa-7089-42f2-92c9-9bbe015b9083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:02.952442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e043bf-1753-495c-9e6b-51a2c4ab67eb None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:02.963465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e043bf-1753-495c-9e6b-51a2c4ab67eb None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:02.970262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414287e5-3f48-4e16-b7a4-ab78f7eadeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:02.971230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414287e5-3f48-4e16-b7a4-ab78f7eadeaf admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.971365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414287e5-3f48-4e16-b7a4-ab78f7eadeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.971610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 765/1548] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:02.971737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414287e5-3f48-4e16-b7a4-ab78f7eadeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.972261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414287e5-3f48-4e16-b7a4-ab78f7eadeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:02.973401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f076d-4b80-4a9c-8807-c352aa1c77f6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.974017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f076d-4b80-4a9c-8807-c352aa1c77f6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.974262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414287e5-3f48-4e16-b7a4-ab78f7eadeaf admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:02.974777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f076d-4b80-4a9c-8807-c352aa1c77f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:02.988212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 784/1549] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:02.989507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71092a27-ada0-462a-9ea4-10b977bc9d43 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:02.991305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71092a27-ada0-462a-9ea4-10b977bc9d43 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:02.991305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71092a27-ada0-462a-9ea4-10b977bc9d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.000534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8e3f076d-4b80-4a9c-8807-c352aa1c77f6 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.004031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8e3f076d-4b80-4a9c-8807-c352aa1c77f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.012538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 766/1550] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.013985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3dba-92d5-41a2-b272-29387b110920 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.016344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3dba-92d5-41a2-b272-29387b110920 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.016344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a3dba-92d5-41a2-b272-29387b110920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.020333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-71092a27-ada0-462a-9ea4-10b977bc9d43 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.021570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-71092a27-ada0-462a-9ea4-10b977bc9d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.029876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 785/1551] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.033093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437be9e-c004-4f19-a746-c1cab6d2ecdf None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.033093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437be9e-c004-4f19-a746-c1cab6d2ecdf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.033093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437be9e-c004-4f19-a746-c1cab6d2ecdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.037585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-df2a3dba-92d5-41a2-b272-29387b110920 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.039012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-df2a3dba-92d5-41a2-b272-29387b110920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.048011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 767/1552] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.048649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a3dba-92d5-41a2-b272-29387b110920 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.052518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6437be9e-c004-4f19-a746-c1cab6d2ecdf None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.053710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6437be9e-c004-4f19-a746-c1cab6d2ecdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.058606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a3dba-92d5-41a2-b272-29387b110920 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.061745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 786/1553] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.062952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437be9e-c004-4f19-a746-c1cab6d2ecdf None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.063806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815285c-dc31-4516-be4b-c9f05766bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.064491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815285c-dc31-4516-be4b-c9f05766bf39 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.064795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815285c-dc31-4516-be4b-c9f05766bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.065138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815285c-dc31-4516-be4b-c9f05766bf39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.065793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815285c-dc31-4516-be4b-c9f05766bf39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:03.066091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815285c-dc31-4516-be4b-c9f05766bf39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.067902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815285c-dc31-4516-be4b-c9f05766bf39 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.072252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437be9e-c004-4f19-a746-c1cab6d2ecdf None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.076829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066ca32-fe5f-4a6a-b979-3fc0783a5034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.078080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066ca32-fe5f-4a6a-b979-3fc0783a5034 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.078614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066ca32-fe5f-4a6a-b979-3fc0783a5034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.079004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066ca32-fe5f-4a6a-b979-3fc0783a5034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.079642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066ca32-fe5f-4a6a-b979-3fc0783a5034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.080745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 768/1554] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:03.081224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8066ca32-fe5f-4a6a-b979-3fc0783a5034 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.081944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a14f-463b-4458-94f3-819f300ae882 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.082226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a14f-463b-4458-94f3-819f300ae882 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.082532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469a14f-463b-4458-94f3-819f300ae882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.091838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 787/1555] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:03.092954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066ca32-fe5f-4a6a-b979-3fc0783a5034 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.103580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2469a14f-463b-4458-94f3-819f300ae882 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.103754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066ca32-fe5f-4a6a-b979-3fc0783a5034 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.104923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2469a14f-463b-4458-94f3-819f300ae882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.108323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e9b37-bc3e-4d46-9920-cd773bf8a24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.109003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e9b37-bc3e-4d46-9920-cd773bf8a24c admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.109272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e9b37-bc3e-4d46-9920-cd773bf8a24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.109542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e9b37-bc3e-4d46-9920-cd773bf8a24c admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.110222 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e9b37-bc3e-4d46-9920-cd773bf8a24c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:03.110537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e9b37-bc3e-4d46-9920-cd773bf8a24c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.111533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WARNING py.warnings [None req-ef5e9b37-bc3e-4d46-9920-cd773bf8a24c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:21:03.111533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: warnings.warn(msg) Sep 09 23:21:03.111533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: Sep 09 23:21:03.112412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e9b37-bc3e-4d46-9920-cd773bf8a24c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.113372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 769/1556] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.114624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb209b71-9e19-4537-9da9-eb93614a4bf4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.114907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb209b71-9e19-4537-9da9-eb93614a4bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.115180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb209b71-9e19-4537-9da9-eb93614a4bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.142328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 788/1557] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:03.142328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624df3a-aec1-42e7-99b4-a57f03f8c4de None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.142328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624df3a-aec1-42e7-99b4-a57f03f8c4de None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.142328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624df3a-aec1-42e7-99b4-a57f03f8c4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.143089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-eb209b71-9e19-4537-9da9-eb93614a4bf4 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.143089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-eb209b71-9e19-4537-9da9-eb93614a4bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.148864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e624df3a-aec1-42e7-99b4-a57f03f8c4de None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.149011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 770/1558] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.150114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1664af9-8bf5-4bd0-8139-11b3c87f8258 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.150387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e624df3a-aec1-42e7-99b4-a57f03f8c4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.150521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1664af9-8bf5-4bd0-8139-11b3c87f8258 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.150681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1664af9-8bf5-4bd0-8139-11b3c87f8258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.159418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 789/1559] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.160328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624df3a-aec1-42e7-99b4-a57f03f8c4de None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.169585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624df3a-aec1-42e7-99b4-a57f03f8c4de None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.171242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e1664af9-8bf5-4bd0-8139-11b3c87f8258 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.172791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e1664af9-8bf5-4bd0-8139-11b3c87f8258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.174244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.174863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.175139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.175408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.176042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:03.176339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.177906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.180676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 771/1560] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.181427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1664af9-8bf5-4bd0-8139-11b3c87f8258 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.189827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 790/1561] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:03.190524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.190804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1664af9-8bf5-4bd0-8139-11b3c87f8258 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.195474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd71f8-beb5-4034-befb-b2cf21cdad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.196070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd71f8-beb5-4034-befb-b2cf21cdad0b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.196347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd71f8-beb5-4034-befb-b2cf21cdad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.196616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd71f8-beb5-4034-befb-b2cf21cdad0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.197239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd71f8-beb5-4034-befb-b2cf21cdad0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.198248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-c7bd71f8-beb5-4034-befb-b2cf21cdad0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:21:03.198248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Sep 09 23:21:03.198248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: Sep 09 23:21:03.199472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd71f8-beb5-4034-befb-b2cf21cdad0b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.202070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683bb51-a818-4e19-8037-5ad8040a36c2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.207560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fad6a-e7cd-49d6-bb5d-995084f8c0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.208208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fad6a-e7cd-49d6-bb5d-995084f8c0fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.208483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fad6a-e7cd-49d6-bb5d-995084f8c0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.208751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445fad6a-e7cd-49d6-bb5d-995084f8c0fb admin admin] PATH_INFO: `/v3/projects/b4291c4cdc1d4f4eae035c4800670101/users/a496b0d69dd24d6980fa7301b7fa2a90/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.225040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fad6a-e7cd-49d6-bb5d-995084f8c0fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4291c4cdc1d4f4eae035c4800670101, user_id=a496b0d69dd24d6980fa7301b7fa2a90, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.228021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445fad6a-e7cd-49d6-bb5d-995084f8c0fb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.362052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 772/1562] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:03.363438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7a8de-fa41-4881-88ea-b814a71eb6c0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.363724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7a8de-fa41-4881-88ea-b814a71eb6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.363992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7a8de-fa41-4881-88ea-b814a71eb6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.378906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 791/1563] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:03 2022] PUT /identity/v3/projects/b4291c4cdc1d4f4eae035c4800670101/users/a496b0d69dd24d6980fa7301b7fa2a90/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:03.379585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fad6a-e7cd-49d6-bb5d-995084f8c0fb admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.384989 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4ea7a8de-fa41-4881-88ea-b814a71eb6c0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.386294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4ea7a8de-fa41-4881-88ea-b814a71eb6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.400290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:03.440288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:03.446699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445fad6a-e7cd-49d6-bb5d-995084f8c0fb admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.451361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a41f2-2e6f-4fb1-9501-9c8e2159ec99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.451513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 773/1564] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.451967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a41f2-2e6f-4fb1-9501-9c8e2159ec99 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.452242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a41f2-2e6f-4fb1-9501-9c8e2159ec99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.452508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a41f2-2e6f-4fb1-9501-9c8e2159ec99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.452756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268105-2d1e-4ccd-b3ee-d465481f0805 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.453041 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268105-2d1e-4ccd-b3ee-d465481f0805 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.453146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a41f2-2e6f-4fb1-9501-9c8e2159ec99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.453309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98268105-2d1e-4ccd-b3ee-d465481f0805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.456817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a41f2-2e6f-4fb1-9501-9c8e2159ec99 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.475479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-98268105-2d1e-4ccd-b3ee-d465481f0805 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.476698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-98268105-2d1e-4ccd-b3ee-d465481f0805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.485383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 774/1565] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.486156 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98268105-2d1e-4ccd-b3ee-d465481f0805 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.495498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98268105-2d1e-4ccd-b3ee-d465481f0805 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.500137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.506026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.506026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.506026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.506026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:03.506026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.506026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.516014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 775/1566] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:03.517538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.527657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba93e8-3ca9-4e5a-8939-49b0190614e1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.532397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.533061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.533334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.533598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.534211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:03.534570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.536105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.547747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 776/1567] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:03.548554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.558273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd35ef4-94b5-4598-b812-53bbe45f0d0f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.562913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4679106-fec5-4752-88e5-f0c84035a4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.564114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4679106-fec5-4752-88e5-f0c84035a4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.564412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4679106-fec5-4752-88e5-f0c84035a4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.564679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4679106-fec5-4752-88e5-f0c84035a4e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.565287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4679106-fec5-4752-88e5-f0c84035a4e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.567243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4679106-fec5-4752-88e5-f0c84035a4e8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.571259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 792/1568] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/projects => generated 403 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:03.571259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a41f2-2e6f-4fb1-9501-9c8e2159ec99 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.634311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a41f2-2e6f-4fb1-9501-9c8e2159ec99 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.639015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.639605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.639878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.640142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.640758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.642304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WARNING py.warnings [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:21:03.642304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: warnings.warn(msg) Sep 09 23:21:03.642304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: Sep 09 23:21:03.643631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.660695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 777/1569] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:03.660695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654854-4272-4c48-ac08-9930580cd451 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.660990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654854-4272-4c48-ac08-9930580cd451 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.661148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4654854-4272-4c48-ac08-9930580cd451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.680295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d4654854-4272-4c48-ac08-9930580cd451 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:03.681506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d4654854-4272-4c48-ac08-9930580cd451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:03.739307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 778/1570] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:03.740146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4654854-4272-4c48-ac08-9930580cd451 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.749070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4654854-4272-4c48-ac08-9930580cd451 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.753877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.754497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.754769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.755043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.755675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:03.755970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.757555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.761162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 793/1571] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/users => generated 282 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:03.766093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.770957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 779/1572] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:03.771646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.772431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a230951-9e92-451f-a737-3fafcdc8173b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.781263 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3ba6e-614b-4eee-af5f-cbfe4e13d6c9 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.781620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.784983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.785259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.785525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.786162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:03.786483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.786887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242641d-7206-430b-87b0-8b09fda49ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.787664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242641d-7206-430b-87b0-8b09fda49ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.787943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242641d-7206-430b-87b0-8b09fda49ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.788144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.788277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242641d-7206-430b-87b0-8b09fda49ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.788822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242641d-7206-430b-87b0-8b09fda49ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.790759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f242641d-7206-430b-87b0-8b09fda49ab1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.800659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 794/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:03.801784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.812173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e8cdc4-622e-4165-a5ef-da9a94a50c4e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.817280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79d1a7-3975-4b2e-9616-5e69f7fd45b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.817864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d1a7-3975-4b2e-9616-5e69f7fd45b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.818141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d1a7-3975-4b2e-9616-5e69f7fd45b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.818452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d1a7-3975-4b2e-9616-5e69f7fd45b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.819065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79d1a7-3975-4b2e-9616-5e69f7fd45b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.820934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79d1a7-3975-4b2e-9616-5e69f7fd45b7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:03.881849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 780/1574] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/projects => generated 409 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:03.885825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242641d-7206-430b-87b0-8b09fda49ab1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.905211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 795/1575] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:03.906138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79d1a7-3975-4b2e-9616-5e69f7fd45b7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:03.955189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242641d-7206-430b-87b0-8b09fda49ab1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.965023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:03.974924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:03.979267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79d1a7-3975-4b2e-9616-5e69f7fd45b7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:03.983893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a1cd8-43b4-4e02-974e-e0fab78a8cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:03.984514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a1cd8-43b4-4e02-974e-e0fab78a8cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:03.984778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a1cd8-43b4-4e02-974e-e0fab78a8cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:03.985099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a1cd8-43b4-4e02-974e-e0fab78a8cbe admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:03.986233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a1cd8-43b4-4e02-974e-e0fab78a8cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:03.988256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a1cd8-43b4-4e02-974e-e0fab78a8cbe admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.013074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06898a9d-0584-45d8-8d79-0a2716f25119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.013683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898a9d-0584-45d8-8d79-0a2716f25119 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.013957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898a9d-0584-45d8-8d79-0a2716f25119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.014224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06898a9d-0584-45d8-8d79-0a2716f25119 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.014879 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06898a9d-0584-45d8-8d79-0a2716f25119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.017417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06898a9d-0584-45d8-8d79-0a2716f25119 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.116781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 796/1576] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/users => generated 284 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:04.116781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a1cd8-43b4-4e02-974e-e0fab78a8cbe admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.168929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a1cd8-43b4-4e02-974e-e0fab78a8cbe admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.173106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 781/1577] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:03 2022] POST /identity/v3/users => generated 302 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:04.173383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad7975-6ecb-4cff-b7a4-7f5e1a5ee041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.173990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06898a9d-0584-45d8-8d79-0a2716f25119 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.174128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad7975-6ecb-4cff-b7a4-7f5e1a5ee041 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.174500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad7975-6ecb-4cff-b7a4-7f5e1a5ee041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.174777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad7975-6ecb-4cff-b7a4-7f5e1a5ee041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.184257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad7975-6ecb-4cff-b7a4-7f5e1a5ee041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.185524 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06898a9d-0584-45d8-8d79-0a2716f25119 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.185889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad7975-6ecb-4cff-b7a4-7f5e1a5ee041 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.190828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652746f-9635-4009-ba7c-31216f16532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.191490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652746f-9635-4009-ba7c-31216f16532a admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.191768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652746f-9635-4009-ba7c-31216f16532a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.192085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652746f-9635-4009-ba7c-31216f16532a admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.192743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652746f-9635-4009-ba7c-31216f16532a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.194367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652746f-9635-4009-ba7c-31216f16532a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.196653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 797/1578] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:04.197513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad7975-6ecb-4cff-b7a4-7f5e1a5ee041 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.208241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 782/1579] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.208241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652746f-9635-4009-ba7c-31216f16532a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.208490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad7975-6ecb-4cff-b7a4-7f5e1a5ee041 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.213533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829053-42d5-45f3-85ef-8c77ef4e364e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.214136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829053-42d5-45f3-85ef-8c77ef4e364e admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.214521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829053-42d5-45f3-85ef-8c77ef4e364e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.214907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6829053-42d5-45f3-85ef-8c77ef4e364e admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.215528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829053-42d5-45f3-85ef-8c77ef4e364e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.216968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652746f-9635-4009-ba7c-31216f16532a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.217478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6829053-42d5-45f3-85ef-8c77ef4e364e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.222139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c32ad-288a-43b8-9b46-041f727e373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.222768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c32ad-288a-43b8-9b46-041f727e373a admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.223043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c32ad-288a-43b8-9b46-041f727e373a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.223313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c32ad-288a-43b8-9b46-041f727e373a admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.223925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c32ad-288a-43b8-9b46-041f727e373a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.225803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c32ad-288a-43b8-9b46-041f727e373a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.353492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 783/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:04 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:04.353492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c32ad-288a-43b8-9b46-041f727e373a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.355490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 798/1581] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:04 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:04.356490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829053-42d5-45f3-85ef-8c77ef4e364e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.424831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c32ad-288a-43b8-9b46-041f727e373a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.429535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78c21b-25f7-4e27-9bfc-49cf389dc359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.430195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78c21b-25f7-4e27-9bfc-49cf389dc359 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.430494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78c21b-25f7-4e27-9bfc-49cf389dc359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.430767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78c21b-25f7-4e27-9bfc-49cf389dc359 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.431423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78c21b-25f7-4e27-9bfc-49cf389dc359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.432803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6829053-42d5-45f3-85ef-8c77ef4e364e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.433384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78c21b-25f7-4e27-9bfc-49cf389dc359 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.437533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bf22e-c08a-4e0a-80c4-6b0c87138cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.438855 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bf22e-c08a-4e0a-80c4-6b0c87138cdb admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.439409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bf22e-c08a-4e0a-80c4-6b0c87138cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.439409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bf22e-c08a-4e0a-80c4-6b0c87138cdb admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.439935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bf22e-c08a-4e0a-80c4-6b0c87138cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.441857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bf22e-c08a-4e0a-80c4-6b0c87138cdb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.579993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 784/1582] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:04 2022] POST /identity/v3/users => generated 305 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:04.579993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78c21b-25f7-4e27-9bfc-49cf389dc359 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.588649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78c21b-25f7-4e27-9bfc-49cf389dc359 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.593332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb997cce-1db7-4b5f-97c1-61d796c69953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.593937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997cce-1db7-4b5f-97c1-61d796c69953 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.594205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997cce-1db7-4b5f-97c1-61d796c69953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.595287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb997cce-1db7-4b5f-97c1-61d796c69953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.596153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb997cce-1db7-4b5f-97c1-61d796c69953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.597853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb997cce-1db7-4b5f-97c1-61d796c69953 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.608684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 785/1583] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.610432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb997cce-1db7-4b5f-97c1-61d796c69953 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.620903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb997cce-1db7-4b5f-97c1-61d796c69953 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.625463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05ef5c-28de-46f9-8bbf-9843fe53f5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.631135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05ef5c-28de-46f9-8bbf-9843fe53f5fe admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.631135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05ef5c-28de-46f9-8bbf-9843fe53f5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.631135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05ef5c-28de-46f9-8bbf-9843fe53f5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.631135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05ef5c-28de-46f9-8bbf-9843fe53f5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.631135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05ef5c-28de-46f9-8bbf-9843fe53f5fe admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.639197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 786/1584] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.639952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05ef5c-28de-46f9-8bbf-9843fe53f5fe admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.649569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05ef5c-28de-46f9-8bbf-9843fe53f5fe admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.654133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018d616-dfba-4f7b-9686-62a2cd0aaf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.654694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018d616-dfba-4f7b-9686-62a2cd0aaf83 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.654966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018d616-dfba-4f7b-9686-62a2cd0aaf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.655232 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018d616-dfba-4f7b-9686-62a2cd0aaf83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.655834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018d616-dfba-4f7b-9686-62a2cd0aaf83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.658154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018d616-dfba-4f7b-9686-62a2cd0aaf83 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.668063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 787/1585] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.668837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018d616-dfba-4f7b-9686-62a2cd0aaf83 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.673056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 799/1586] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:04 2022] POST /identity/v3/users => generated 289 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:04.673056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bf22e-c08a-4e0a-80c4-6b0c87138cdb admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.679053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018d616-dfba-4f7b-9686-62a2cd0aaf83 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.681830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bf22e-c08a-4e0a-80c4-6b0c87138cdb admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.683840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee775c9-5f24-4815-b400-ef98da919cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.684559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee775c9-5f24-4815-b400-ef98da919cda admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.684839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee775c9-5f24-4815-b400-ef98da919cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.685109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee775c9-5f24-4815-b400-ef98da919cda admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.685729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee775c9-5f24-4815-b400-ef98da919cda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.686636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b0dbc-046a-47e7-8322-2bd63239acb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.687216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0dbc-046a-47e7-8322-2bd63239acb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.687493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0dbc-046a-47e7-8322-2bd63239acb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.687706 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee775c9-5f24-4815-b400-ef98da919cda admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.687820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b0dbc-046a-47e7-8322-2bd63239acb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.688656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b0dbc-046a-47e7-8322-2bd63239acb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.690624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b0dbc-046a-47e7-8322-2bd63239acb7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.817496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 788/1587] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:04 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:04.818546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee775c9-5f24-4815-b400-ef98da919cda admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.835175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee775c9-5f24-4815-b400-ef98da919cda admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.840352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4e9e4-a62b-4541-902f-5f9a9f52765e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.841022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e9e4-a62b-4541-902f-5f9a9f52765e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.844440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e9e4-a62b-4541-902f-5f9a9f52765e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.844440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4e9e4-a62b-4541-902f-5f9a9f52765e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.844440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4e9e4-a62b-4541-902f-5f9a9f52765e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.844440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4e9e4-a62b-4541-902f-5f9a9f52765e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.846393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:04.855783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 789/1588] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.857061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4e9e4-a62b-4541-902f-5f9a9f52765e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.868314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4e9e4-a62b-4541-902f-5f9a9f52765e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.873237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df0019-1efa-4582-a384-c9074ea3f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.873923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df0019-1efa-4582-a384-c9074ea3f6ae admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.874202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df0019-1efa-4582-a384-c9074ea3f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.874501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df0019-1efa-4582-a384-c9074ea3f6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.875131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df0019-1efa-4582-a384-c9074ea3f6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.877350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df0019-1efa-4582-a384-c9074ea3f6ae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.887731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 790/1589] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.888552 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df0019-1efa-4582-a384-c9074ea3f6ae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.898363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df0019-1efa-4582-a384-c9074ea3f6ae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.903188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329b29-7d6b-495c-b7e3-59dc41aaf13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.903769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329b29-7d6b-495c-b7e3-59dc41aaf13e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.904043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329b29-7d6b-495c-b7e3-59dc41aaf13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.904308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329b29-7d6b-495c-b7e3-59dc41aaf13e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.904928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329b29-7d6b-495c-b7e3-59dc41aaf13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.906997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329b29-7d6b-495c-b7e3-59dc41aaf13e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.917267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 791/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.918539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329b29-7d6b-495c-b7e3-59dc41aaf13e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.925124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 800/1591] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:04 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:04.925124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b0dbc-046a-47e7-8322-2bd63239acb7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.928869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329b29-7d6b-495c-b7e3-59dc41aaf13e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.933680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.934270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b0dbc-046a-47e7-8322-2bd63239acb7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.934635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.935006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.935340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] PATH_INFO: `/v3/projects/bef846fcbf2442c1b35a7a98fc782fcf/users/989f2baf667d4ae5b1c176f3aa0ac3b4/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.939616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f903-96af-4c6e-be9d-6d03f39b8516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.940351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f903-96af-4c6e-be9d-6d03f39b8516 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.940649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f903-96af-4c6e-be9d-6d03f39b8516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.940894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f903-96af-4c6e-be9d-6d03f39b8516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.941520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f903-96af-4c6e-be9d-6d03f39b8516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.943139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f903-96af-4c6e-be9d-6d03f39b8516 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.951969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef846fcbf2442c1b35a7a98fc782fcf, user_id=989f2baf667d4ae5b1c176f3aa0ac3b4, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.953040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:21:04.953040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Sep 09 23:21:04.953040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: Sep 09 23:21:04.958617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 801/1592] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.958780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.958867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f903-96af-4c6e-be9d-6d03f39b8516 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.967576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f903-96af-4c6e-be9d-6d03f39b8516 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:04.974490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038c077-4677-41a6-8613-d15795ef663e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:04.974490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038c077-4677-41a6-8613-d15795ef663e admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:04.974490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038c077-4677-41a6-8613-d15795ef663e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:04.974490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038c077-4677-41a6-8613-d15795ef663e admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:04.974920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038c077-4677-41a6-8613-d15795ef663e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:04.976459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038c077-4677-41a6-8613-d15795ef663e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:04.987034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 802/1593] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:04.987755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038c077-4677-41a6-8613-d15795ef663e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:04.997102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038c077-4677-41a6-8613-d15795ef663e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.003814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa0c6b-6e2d-4f55-a006-dc646b9015fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.014535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa0c6b-6e2d-4f55-a006-dc646b9015fe admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.014535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa0c6b-6e2d-4f55-a006-dc646b9015fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.014535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa0c6b-6e2d-4f55-a006-dc646b9015fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.014535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa0c6b-6e2d-4f55-a006-dc646b9015fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.014535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa0c6b-6e2d-4f55-a006-dc646b9015fe admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.017774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 803/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:05.018544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa0c6b-6e2d-4f55-a006-dc646b9015fe admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.027804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa0c6b-6e2d-4f55-a006-dc646b9015fe admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.032392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19aa23-7d3d-471b-9347-4f6ae455e503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.033139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19aa23-7d3d-471b-9347-4f6ae455e503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.033292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19aa23-7d3d-471b-9347-4f6ae455e503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.033567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19aa23-7d3d-471b-9347-4f6ae455e503 admin admin] PATH_INFO: `/v3/projects/5819879d09b8412e8127cd03ca90529a/users/613682a6b93b4a60aaf25a7c72995fe1/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.035845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:05.048792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19aa23-7d3d-471b-9347-4f6ae455e503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5819879d09b8412e8127cd03ca90529a, user_id=613682a6b93b4a60aaf25a7c72995fe1, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.051461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19aa23-7d3d-471b-9347-4f6ae455e503 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.069687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 792/1595] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:04 2022] PUT /identity/v3/projects/bef846fcbf2442c1b35a7a98fc782fcf/users/989f2baf667d4ae5b1c176f3aa0ac3b4/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:05.070701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.133341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 804/1596] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:05 2022] PUT /identity/v3/projects/5819879d09b8412e8127cd03ca90529a/users/613682a6b93b4a60aaf25a7c72995fe1/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:05.139184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19aa23-7d3d-471b-9347-4f6ae455e503 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.167126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ac9e6-4378-4e86-8590-7ff67bd6e2e3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.200309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19aa23-7d3d-471b-9347-4f6ae455e503 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.206376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe45e3-192f-47db-a37f-dd113ceae143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.208305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe45e3-192f-47db-a37f-dd113ceae143 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.208305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe45e3-192f-47db-a37f-dd113ceae143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.208305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe45e3-192f-47db-a37f-dd113ceae143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.208537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe45e3-192f-47db-a37f-dd113ceae143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.209919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe45e3-192f-47db-a37f-dd113ceae143 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.220416 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 805/1597] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:05.221001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe45e3-192f-47db-a37f-dd113ceae143 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.231343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe45e3-192f-47db-a37f-dd113ceae143 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.234174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b4567-9662-49cd-94b5-36d67332733c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.235030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4567-9662-49cd-94b5-36d67332733c admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.236081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4567-9662-49cd-94b5-36d67332733c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.236381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac3ea2-d1cc-4de7-b362-b6431e62ef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.236613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4567-9662-49cd-94b5-36d67332733c admin admin] PATH_INFO: `/v3/projects/d611d0dcf35842838e4b6acea505a622/users/42cede68d7fd414e991916c17a262385/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.236976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3ea2-d1cc-4de7-b362-b6431e62ef83 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.243965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3ea2-d1cc-4de7-b362-b6431e62ef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.243965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac3ea2-d1cc-4de7-b362-b6431e62ef83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.243965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac3ea2-d1cc-4de7-b362-b6431e62ef83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.243965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac3ea2-d1cc-4de7-b362-b6431e62ef83 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.250263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 806/1598] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:05.251670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac3ea2-d1cc-4de7-b362-b6431e62ef83 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.253873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b4567-9662-49cd-94b5-36d67332733c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d611d0dcf35842838e4b6acea505a622, user_id=42cede68d7fd414e991916c17a262385, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.256761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b4567-9662-49cd-94b5-36d67332733c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.261662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac3ea2-d1cc-4de7-b362-b6431e62ef83 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.267186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944023e1-1739-46c2-9d36-af346ecf041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.267942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944023e1-1739-46c2-9d36-af346ecf041e admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.268274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944023e1-1739-46c2-9d36-af346ecf041e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.268544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944023e1-1739-46c2-9d36-af346ecf041e admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.269174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944023e1-1739-46c2-9d36-af346ecf041e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.270766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944023e1-1739-46c2-9d36-af346ecf041e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.281143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 807/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:05.282617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944023e1-1739-46c2-9d36-af346ecf041e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.291713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944023e1-1739-46c2-9d36-af346ecf041e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.296918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e354f75-981e-4847-8e9a-1ab7d3aac64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.298166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e354f75-981e-4847-8e9a-1ab7d3aac64b admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.298476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e354f75-981e-4847-8e9a-1ab7d3aac64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.298822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e354f75-981e-4847-8e9a-1ab7d3aac64b admin admin] PATH_INFO: `/v3/projects/191adf2320154d4297c2dc5f6cf23de8/users/c25910b8d0554a7bbac3ed1450bb62bc/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.312565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e354f75-981e-4847-8e9a-1ab7d3aac64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=191adf2320154d4297c2dc5f6cf23de8, user_id=c25910b8d0554a7bbac3ed1450bb62bc, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.315387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e354f75-981e-4847-8e9a-1ab7d3aac64b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.360893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 793/1600] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:05 2022] PUT /identity/v3/projects/d611d0dcf35842838e4b6acea505a622/users/42cede68d7fd414e991916c17a262385/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:05.362357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed799ab-4b8d-4323-a8dc-19672748f8e4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.362646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed799ab-4b8d-4323-a8dc-19672748f8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.363051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed799ab-4b8d-4323-a8dc-19672748f8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.392937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bed799ab-4b8d-4323-a8dc-19672748f8e4 None None] MFA Rules not processed for user `989f2baf667d4ae5b1c176f3aa0ac3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:05.403115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bed799ab-4b8d-4323-a8dc-19672748f8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:05.403280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 808/1601] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:05 2022] PUT /identity/v3/projects/191adf2320154d4297c2dc5f6cf23de8/users/c25910b8d0554a7bbac3ed1450bb62bc/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:05.403280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a0e0-f6f3-45d1-bb99-6f1173ec36b1 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.403280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a0e0-f6f3-45d1-bb99-6f1173ec36b1 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.403280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a0e0-f6f3-45d1-bb99-6f1173ec36b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.439555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:05.441950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6bf6a0e0-f6f3-45d1-bb99-6f1173ec36b1 None None] MFA Rules not processed for user `613682a6b93b4a60aaf25a7c72995fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:05.443712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6bf6a0e0-f6f3-45d1-bb99-6f1173ec36b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:05.479870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 794/1602] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:05.480987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed799ab-4b8d-4323-a8dc-19672748f8e4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.521931 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 809/1603] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:05.526808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a0e0-f6f3-45d1-bb99-6f1173ec36b1 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.547667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed799ab-4b8d-4323-a8dc-19672748f8e4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.555821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:05.555963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435c0c-6b73-4047-b36d-f93305080603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.555963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435c0c-6b73-4047-b36d-f93305080603 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.555963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435c0c-6b73-4047-b36d-f93305080603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.555963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7435c0c-6b73-4047-b36d-f93305080603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.556480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435c0c-6b73-4047-b36d-f93305080603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.558088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7435c0c-6b73-4047-b36d-f93305080603 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.568888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 795/1604] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:05.569678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435c0c-6b73-4047-b36d-f93305080603 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.579952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7435c0c-6b73-4047-b36d-f93305080603 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.584604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7ea9b-a97a-4189-876d-521a5537e0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.585238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7ea9b-a97a-4189-876d-521a5537e0e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.585514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7ea9b-a97a-4189-876d-521a5537e0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.585783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7ea9b-a97a-4189-876d-521a5537e0e0 admin admin] PATH_INFO: `/v3/projects/57759aec45f74a689727fca5fe201e40/users/987d2f6bb1ff4f1fa248c3ff410c8119/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.591545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a0e0-f6f3-45d1-bb99-6f1173ec36b1 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.596498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ddcbf6-99d8-4beb-9626-73f349f9a018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.601663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddcbf6-99d8-4beb-9626-73f349f9a018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.601663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddcbf6-99d8-4beb-9626-73f349f9a018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.601663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ddcbf6-99d8-4beb-9626-73f349f9a018 admin admin] PATH_INFO: `/v3/projects/5c3517b03576446a902816ab0bbff2e7/users/f097c4fc270748cc8d41d7bcaba672a7/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.602108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7ea9b-a97a-4189-876d-521a5537e0e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57759aec45f74a689727fca5fe201e40, user_id=987d2f6bb1ff4f1fa248c3ff410c8119, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.603932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a7ea9b-a97a-4189-876d-521a5537e0e0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.613635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddcbf6-99d8-4beb-9626-73f349f9a018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3517b03576446a902816ab0bbff2e7, user_id=f097c4fc270748cc8d41d7bcaba672a7, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.616326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ddcbf6-99d8-4beb-9626-73f349f9a018 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.688586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 796/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:05 2022] PUT /identity/v3/projects/57759aec45f74a689727fca5fe201e40/users/987d2f6bb1ff4f1fa248c3ff410c8119/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:05.688586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5167ec0-7c5c-499b-be00-c54f514127da None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.688586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5167ec0-7c5c-499b-be00-c54f514127da None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.688586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5167ec0-7c5c-499b-be00-c54f514127da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.707362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a5167ec0-7c5c-499b-be00-c54f514127da None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:05.708596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a5167ec0-7c5c-499b-be00-c54f514127da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:05.716808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 810/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:05 2022] PUT /identity/v3/projects/5c3517b03576446a902816ab0bbff2e7/users/f097c4fc270748cc8d41d7bcaba672a7/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:05.722372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545041e1-a5ad-4270-9264-03eadc1d8ffc None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.722372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545041e1-a5ad-4270-9264-03eadc1d8ffc None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.722372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545041e1-a5ad-4270-9264-03eadc1d8ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.750927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-545041e1-a5ad-4270-9264-03eadc1d8ffc None None] MFA Rules not processed for user `c25910b8d0554a7bbac3ed1450bb62bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:05.753025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-545041e1-a5ad-4270-9264-03eadc1d8ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:05.787926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 797/1607] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:05.789049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5167ec0-7c5c-499b-be00-c54f514127da None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.804120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5167ec0-7c5c-499b-be00-c54f514127da None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.810850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290b295-f5a3-4903-961a-dcfa9c6233fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.812258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290b295-f5a3-4903-961a-dcfa9c6233fa admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.814184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290b295-f5a3-4903-961a-dcfa9c6233fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.814528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290b295-f5a3-4903-961a-dcfa9c6233fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.815180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290b295-f5a3-4903-961a-dcfa9c6233fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.817764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7290b295-f5a3-4903-961a-dcfa9c6233fa admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.832909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 811/1608] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:05 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:05.834046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545041e1-a5ad-4270-9264-03eadc1d8ffc None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:05.898123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545041e1-a5ad-4270-9264-03eadc1d8ffc None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:05.902389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab61d91-1c54-46ca-bd81-1dbda914cc72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:05.903019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab61d91-1c54-46ca-bd81-1dbda914cc72 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:05.903373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab61d91-1c54-46ca-bd81-1dbda914cc72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:05.903650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab61d91-1c54-46ca-bd81-1dbda914cc72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:05.964023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:05.975775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab61d91-1c54-46ca-bd81-1dbda914cc72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:05.977835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab61d91-1c54-46ca-bd81-1dbda914cc72 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:05.979992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 798/1609] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:05 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:05.980716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290b295-f5a3-4903-961a-dcfa9c6233fa admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.010058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 812/1610] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:06.010058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ca730-eab8-4fb3-91a2-de3ac4721fb0 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.010058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ca730-eab8-4fb3-91a2-de3ac4721fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.010851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:06.011841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ca730-eab8-4fb3-91a2-de3ac4721fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.037087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5a7ca730-eab8-4fb3-91a2-de3ac4721fb0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:06.038590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5a7ca730-eab8-4fb3-91a2-de3ac4721fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:06.055515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290b295-f5a3-4903-961a-dcfa9c6233fa admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.062586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2ac24-eddd-4117-b794-f8e0ebe1d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.063459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2ac24-eddd-4117-b794-f8e0ebe1d010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.067780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2ac24-eddd-4117-b794-f8e0ebe1d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.067780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2ac24-eddd-4117-b794-f8e0ebe1d010 admin admin] PATH_INFO: `/v3/projects/d9dc686533584d00b764ff90330a6d0b/users/f29c9543ad85449cacc674d92fba578b/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.093266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2ac24-eddd-4117-b794-f8e0ebe1d010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9dc686533584d00b764ff90330a6d0b, user_id=f29c9543ad85449cacc674d92fba578b, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.097417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2ac24-eddd-4117-b794-f8e0ebe1d010 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.127338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 813/1611] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:06.133786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3856504-0d69-48c5-a407-2c591bf67623 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.134076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3856504-0d69-48c5-a407-2c591bf67623 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.134362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3856504-0d69-48c5-a407-2c591bf67623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.171681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a3856504-0d69-48c5-a407-2c591bf67623 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:06.178514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 799/1612] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:05 2022] PUT /identity/v3/projects/d9dc686533584d00b764ff90330a6d0b/users/f29c9543ad85449cacc674d92fba578b/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:06.178514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2ac24-eddd-4117-b794-f8e0ebe1d010 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.178811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a3856504-0d69-48c5-a407-2c591bf67623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:06.254291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2ac24-eddd-4117-b794-f8e0ebe1d010 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.261785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e881266-fdd3-415a-a8d2-16462dee4a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.262865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e881266-fdd3-415a-a8d2-16462dee4a6d service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.267134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e881266-fdd3-415a-a8d2-16462dee4a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.267134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e881266-fdd3-415a-a8d2-16462dee4a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.303330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 814/1613] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:06.303330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3856504-0d69-48c5-a407-2c591bf67623 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.319827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3856504-0d69-48c5-a407-2c591bf67623 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.331482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5578-1d07-40b4-9713-a1dc067ef1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.332877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5578-1d07-40b4-9713-a1dc067ef1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.333186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5578-1d07-40b4-9713-a1dc067ef1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.333448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5578-1d07-40b4-9713-a1dc067ef1f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.335209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e5578-1d07-40b4-9713-a1dc067ef1f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.337199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e5578-1d07-40b4-9713-a1dc067ef1f1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.366303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e881266-fdd3-415a-a8d2-16462dee4a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.367805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e881266-fdd3-415a-a8d2-16462dee4a6d service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.378217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 800/1614] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:06.379220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e881266-fdd3-415a-a8d2-16462dee4a6d service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.403406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e881266-fdd3-415a-a8d2-16462dee4a6d service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.412141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff696a-f06b-4dcc-bd4e-0455410b2e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.412948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff696a-f06b-4dcc-bd4e-0455410b2e5a admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.413865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff696a-f06b-4dcc-bd4e-0455410b2e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.414143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff696a-f06b-4dcc-bd4e-0455410b2e5a admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.414916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff696a-f06b-4dcc-bd4e-0455410b2e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.417257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff696a-f06b-4dcc-bd4e-0455410b2e5a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.468279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 815/1615] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:06 2022] POST /identity/v3/projects => generated 409 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:06.471711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5578-1d07-40b4-9713-a1dc067ef1f1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.554017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5578-1d07-40b4-9713-a1dc067ef1f1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.558518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69531-0972-4c35-ad03-406f1d9aad42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.559129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69531-0972-4c35-ad03-406f1d9aad42 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.559674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69531-0972-4c35-ad03-406f1d9aad42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.560015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69531-0972-4c35-ad03-406f1d9aad42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.561707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 801/1616] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:06 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:06.563664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eab938-7c36-49f7-91f0-b39a6a94f780 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.564002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eab938-7c36-49f7-91f0-b39a6a94f780 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.564777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eab938-7c36-49f7-91f0-b39a6a94f780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.589338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:06.631916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-81eab938-7c36-49f7-91f0-b39a6a94f780 None None] MFA Rules not processed for user `f29c9543ad85449cacc674d92fba578b`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:06.635871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-81eab938-7c36-49f7-91f0-b39a6a94f780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:06.643485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa69531-0972-4c35-ad03-406f1d9aad42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.645000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa69531-0972-4c35-ad03-406f1d9aad42 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.670942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 816/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:06.674103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69531-0972-4c35-ad03-406f1d9aad42 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.677748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:06.697522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69531-0972-4c35-ad03-406f1d9aad42 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.704108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86eddf-e175-46a0-8297-b51a8b5abd80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.704807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86eddf-e175-46a0-8297-b51a8b5abd80 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.705331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86eddf-e175-46a0-8297-b51a8b5abd80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.705694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86eddf-e175-46a0-8297-b51a8b5abd80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.715716 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86eddf-e175-46a0-8297-b51a8b5abd80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.717189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86eddf-e175-46a0-8297-b51a8b5abd80 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.740007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 817/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:06.741049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86eddf-e175-46a0-8297-b51a8b5abd80 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.747413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 802/1619] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:06.752848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eab938-7c36-49f7-91f0-b39a6a94f780 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.754192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86eddf-e175-46a0-8297-b51a8b5abd80 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.760013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74b6e6-6812-40e6-bab5-b5d096a6f4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.760995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b6e6-6812-40e6-bab5-b5d096a6f4bb admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.761419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b6e6-6812-40e6-bab5-b5d096a6f4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.762639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b6e6-6812-40e6-bab5-b5d096a6f4bb admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.763408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74b6e6-6812-40e6-bab5-b5d096a6f4bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.765648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74b6e6-6812-40e6-bab5-b5d096a6f4bb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.798521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eab938-7c36-49f7-91f0-b39a6a94f780 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.807003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3b8d0-0722-41e6-a4dc-9fd53b258aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.808168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3b8d0-0722-41e6-a4dc-9fd53b258aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.809026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3b8d0-0722-41e6-a4dc-9fd53b258aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.809567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3b8d0-0722-41e6-a4dc-9fd53b258aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.811053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3b8d0-0722-41e6-a4dc-9fd53b258aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.813649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3b8d0-0722-41e6-a4dc-9fd53b258aa0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.841840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 803/1620] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:06.843986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6baaa7-4998-4a1c-8f2f-649554ca17a4 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.845995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6baaa7-4998-4a1c-8f2f-649554ca17a4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.846951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6baaa7-4998-4a1c-8f2f-649554ca17a4 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.855734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 804/1621] 10.222.0.20 () {54 vars in 962 bytes} [Fri Sep 9 23:21:06 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:06.859360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6baaa7-4998-4a1c-8f2f-649554ca17a4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:06.880666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6baaa7-4998-4a1c-8f2f-649554ca17a4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:06.885920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79267e3-19bb-480a-b0e9-534669d6a662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:06.886915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79267e3-19bb-480a-b0e9-534669d6a662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.893979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79267e3-19bb-480a-b0e9-534669d6a662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.893979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79267e3-19bb-480a-b0e9-534669d6a662 admin admin] PATH_INFO: `/v3/projects/31b13b9b8e45421484add01a80144412/users/4dd7009df75345da82e7fe63d537ebd7/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.906553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79267e3-19bb-480a-b0e9-534669d6a662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b13b9b8e45421484add01a80144412, user_id=4dd7009df75345da82e7fe63d537ebd7, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:06.910087 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79267e3-19bb-480a-b0e9-534669d6a662 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:06.928308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 818/1622] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:06 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:06.928308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa41585-fd9e-4be8-8845-4604a79968d4 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:06.928308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa41585-fd9e-4be8-8845-4604a79968d4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:06.928308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa41585-fd9e-4be8-8845-4604a79968d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:06.993257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-cfa41585-fd9e-4be8-8845-4604a79968d4 None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:06.999981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cfa41585-fd9e-4be8-8845-4604a79968d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:07.025769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 805/1623] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:06 2022] PUT /identity/v3/projects/31b13b9b8e45421484add01a80144412/users/4dd7009df75345da82e7fe63d537ebd7/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:07.026669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79267e3-19bb-480a-b0e9-534669d6a662 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:07.137871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 819/1624] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Sep 9 23:21:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:07.137871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa41585-fd9e-4be8-8845-4604a79968d4 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:07.173454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa41585-fd9e-4be8-8845-4604a79968d4 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:07.209001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79267e3-19bb-480a-b0e9-534669d6a662 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:07.223040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a7841-e3f7-45bc-8901-7dec09b0ad60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:07.224001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7841-e3f7-45bc-8901-7dec09b0ad60 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.230217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7841-e3f7-45bc-8901-7dec09b0ad60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.230789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a7841-e3f7-45bc-8901-7dec09b0ad60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.236089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a7841-e3f7-45bc-8901-7dec09b0ad60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:07.236089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a7841-e3f7-45bc-8901-7dec09b0ad60 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:07.254951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 806/1625] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:07.256024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a7841-e3f7-45bc-8901-7dec09b0ad60 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:07.320745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa7d49-adea-4841-b935-9af7a0f14445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:07.322934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa7d49-adea-4841-b935-9af7a0f14445 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.322934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa7d49-adea-4841-b935-9af7a0f14445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.322934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa7d49-adea-4841-b935-9af7a0f14445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.322934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa7d49-adea-4841-b935-9af7a0f14445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:07.324338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa7d49-adea-4841-b935-9af7a0f14445 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:07.354386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 820/1626] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:07.355419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa7d49-adea-4841-b935-9af7a0f14445 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:07.373706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:07.386048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa7d49-adea-4841-b935-9af7a0f14445 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:07.401400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f75f1e-3c11-4d1d-885c-8cbc56418f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:07.402079 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f75f1e-3c11-4d1d-885c-8cbc56418f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.402393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f75f1e-3c11-4d1d-885c-8cbc56418f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.402661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f75f1e-3c11-4d1d-885c-8cbc56418f6a admin admin] PATH_INFO: `/v3/projects/7e6ad4ba8591498d824b56ac52cb2725/users/aa421da9eff24e96ab0e606a5d20d121/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.421970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a7841-e3f7-45bc-8901-7dec09b0ad60 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:07.429645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43ba6-2d45-495d-a753-8f2e60eb977c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:07.430282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43ba6-2d45-495d-a753-8f2e60eb977c service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.432831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43ba6-2d45-495d-a753-8f2e60eb977c service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.433010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43ba6-2d45-495d-a753-8f2e60eb977c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.444777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f75f1e-3c11-4d1d-885c-8cbc56418f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ad4ba8591498d824b56ac52cb2725, user_id=aa421da9eff24e96ab0e606a5d20d121, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:07.452122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f75f1e-3c11-4d1d-885c-8cbc56418f6a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:07.480552 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:07.522960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a43ba6-2d45-495d-a753-8f2e60eb977c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:07.525151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a43ba6-2d45-495d-a753-8f2e60eb977c service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:07.540102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 807/1627] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:07.540960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43ba6-2d45-495d-a753-8f2e60eb977c service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:07.550481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 821/1628] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:07 2022] PUT /identity/v3/projects/7e6ad4ba8591498d824b56ac52cb2725/users/aa421da9eff24e96ab0e606a5d20d121/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:07.554663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f75f1e-3c11-4d1d-885c-8cbc56418f6a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:07.555789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43ba6-2d45-495d-a753-8f2e60eb977c service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:07.623407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bae86-89cf-48ea-9ed5-6a1d2f90a56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:07.624104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bae86-89cf-48ea-9ed5-6a1d2f90a56f admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.624449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bae86-89cf-48ea-9ed5-6a1d2f90a56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.624872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bae86-89cf-48ea-9ed5-6a1d2f90a56f admin admin] PATH_INFO: `/v3/projects/31b13b9b8e45421484add01a80144412/users/4dd7009df75345da82e7fe63d537ebd7/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.627181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bae86-89cf-48ea-9ed5-6a1d2f90a56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b13b9b8e45421484add01a80144412, user_id=4dd7009df75345da82e7fe63d537ebd7, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:07.630746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bae86-89cf-48ea-9ed5-6a1d2f90a56f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:07.643920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 808/1629] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:07 2022] PUT /identity/v3/projects/31b13b9b8e45421484add01a80144412/users/4dd7009df75345da82e7fe63d537ebd7/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:07.658640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dd22c-018e-43c4-a90d-4241f112fa00 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.660019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dd22c-018e-43c4-a90d-4241f112fa00 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.660517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55dd22c-018e-43c4-a90d-4241f112fa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.688941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f75f1e-3c11-4d1d-885c-8cbc56418f6a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:07.709528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d55dd22c-018e-43c4-a90d-4241f112fa00 None None] MFA Rules not processed for user `4dd7009df75345da82e7fe63d537ebd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:07.711295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d55dd22c-018e-43c4-a90d-4241f112fa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:07.785995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4cfea-524f-439b-b7d4-dce037b6952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:07.787136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4cfea-524f-439b-b7d4-dce037b6952a admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.787725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4cfea-524f-439b-b7d4-dce037b6952a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.788289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4cfea-524f-439b-b7d4-dce037b6952a admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.793229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4cfea-524f-439b-b7d4-dce037b6952a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:07.795365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f4cfea-524f-439b-b7d4-dce037b6952a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:07.814419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 822/1630] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:07.816900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4cfea-524f-439b-b7d4-dce037b6952a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:07.830506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 809/1631] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:07.839112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f4cfea-524f-439b-b7d4-dce037b6952a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:07.851909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198503e-f677-4177-9c3d-ff6208082746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:07.852867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198503e-f677-4177-9c3d-ff6208082746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.853297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198503e-f677-4177-9c3d-ff6208082746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.853571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198503e-f677-4177-9c3d-ff6208082746 admin admin] PATH_INFO: `/v3/projects/7e6ad4ba8591498d824b56ac52cb2725/users/aa421da9eff24e96ab0e606a5d20d121/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.856104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198503e-f677-4177-9c3d-ff6208082746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ad4ba8591498d824b56ac52cb2725, user_id=aa421da9eff24e96ab0e606a5d20d121, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:07.859244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198503e-f677-4177-9c3d-ff6208082746 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:07.864346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55dd22c-018e-43c4-a90d-4241f112fa00 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:07.902450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 823/1632] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:07 2022] PUT /identity/v3/projects/7e6ad4ba8591498d824b56ac52cb2725/users/aa421da9eff24e96ab0e606a5d20d121/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:07.902450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef348-596f-4285-8d62-5f9add7decb3 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.902450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef348-596f-4285-8d62-5f9add7decb3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.902841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ef348-596f-4285-8d62-5f9add7decb3 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.905930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 824/1633] 10.222.0.20 () {54 vars in 912 bytes} [Fri Sep 9 23:21:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:07.908226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf045f3-2c73-4065-9b2b-01e48a1d3f27 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:07.911351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf045f3-2c73-4065-9b2b-01e48a1d3f27 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:07.911604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf045f3-2c73-4065-9b2b-01e48a1d3f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:07.966503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-baf045f3-2c73-4065-9b2b-01e48a1d3f27 None None] MFA Rules not processed for user `aa421da9eff24e96ab0e606a5d20d121`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:07.968252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-baf045f3-2c73-4065-9b2b-01e48a1d3f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:07.971648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55dd22c-018e-43c4-a90d-4241f112fa00 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:08.052403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9aa81-1dd4-4c6b-bc67-f6a3088a50ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:08.054006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9aa81-1dd4-4c6b-bc67-f6a3088a50ca service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:08.054500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9aa81-1dd4-4c6b-bc67-f6a3088a50ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:08.055330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9aa81-1dd4-4c6b-bc67-f6a3088a50ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:08.083392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 825/1634] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:07 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:08.084739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1704e-4f1a-424a-b70f-09071e3f4a38 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:08.085274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1704e-4f1a-424a-b70f-09071e3f4a38 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:08.085549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1704e-4f1a-424a-b70f-09071e3f4a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:08.122980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-74c1704e-4f1a-424a-b70f-09071e3f4a38 None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:08.124834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-74c1704e-4f1a-424a-b70f-09071e3f4a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:08.161510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:08.178708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9aa81-1dd4-4c6b-bc67-f6a3088a50ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:08.180112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9aa81-1dd4-4c6b-bc67-f6a3088a50ca service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:08.210666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 810/1635] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:08.212479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9aa81-1dd4-4c6b-bc67-f6a3088a50ca service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:08.234981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 826/1636] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:21:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:08.239847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9aa81-1dd4-4c6b-bc67-f6a3088a50ca service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:08.246802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d15e5-37f5-4014-8cba-1b3b418d3561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:08.252409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d15e5-37f5-4014-8cba-1b3b418d3561 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:08.252702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d15e5-37f5-4014-8cba-1b3b418d3561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:08.252976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d15e5-37f5-4014-8cba-1b3b418d3561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:08.535939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d15e5-37f5-4014-8cba-1b3b418d3561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:08.535939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d15e5-37f5-4014-8cba-1b3b418d3561 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:08.535939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 811/1637] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:08 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:08.538486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1704e-4f1a-424a-b70f-09071e3f4a38 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:08.538486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1704e-4f1a-424a-b70f-09071e3f4a38 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:08.538486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ff7ac-184b-4649-86c1-60200fd5974a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:08.538486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ff7ac-184b-4649-86c1-60200fd5974a service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:08.540057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ff7ac-184b-4649-86c1-60200fd5974a service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:08.540057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ff7ac-184b-4649-86c1-60200fd5974a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:08.540057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ff7ac-184b-4649-86c1-60200fd5974a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:08.540057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ff7ac-184b-4649-86c1-60200fd5974a service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:08.540057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 827/1638] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:08.598885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:09.010905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1d382-0abe-42d1-8910-2b7dc3bbf8e7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:09.011486 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1d382-0abe-42d1-8910-2b7dc3bbf8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:09.011486 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1d382-0abe-42d1-8910-2b7dc3bbf8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:09.071738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ccb1d382-0abe-42d1-8910-2b7dc3bbf8e7 None None] MFA Rules not processed for user `42cede68d7fd414e991916c17a262385`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:09.074384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ccb1d382-0abe-42d1-8910-2b7dc3bbf8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:09.226481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 812/1639] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:09 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:09.227934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ff7ac-184b-4649-86c1-60200fd5974a service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:09.247912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1d382-0abe-42d1-8910-2b7dc3bbf8e7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:09.259092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1d382-0abe-42d1-8910-2b7dc3bbf8e7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:09.263565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe5a0-6598-419d-ab7a-a2b6fb1b3d8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:09.264181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe5a0-6598-419d-ab7a-a2b6fb1b3d8d service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:09.264463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe5a0-6598-419d-ab7a-a2b6fb1b3d8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:09.264752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dfe5a0-6598-419d-ab7a-a2b6fb1b3d8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:09.345179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfe5a0-6598-419d-ab7a-a2b6fb1b3d8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:09.346663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dfe5a0-6598-419d-ab7a-a2b6fb1b3d8d service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:09.375088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 813/1640] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:09 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:09.386632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ff7ac-184b-4649-86c1-60200fd5974a service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:09.407336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cb6ea-4058-4ea3-a88a-d4c88ba31c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:09.407974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cb6ea-4058-4ea3-a88a-d4c88ba31c9a admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:09.408250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cb6ea-4058-4ea3-a88a-d4c88ba31c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:09.408600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cb6ea-4058-4ea3-a88a-d4c88ba31c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:09.409168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cb6ea-4058-4ea3-a88a-d4c88ba31c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:09.419137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cb6ea-4058-4ea3-a88a-d4c88ba31c9a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:09.582117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 828/1641] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:09 2022] POST /identity/v3/projects => generated 403 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:09.586684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe5a0-6598-419d-ab7a-a2b6fb1b3d8d service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:09.602918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dfe5a0-6598-419d-ab7a-a2b6fb1b3d8d service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:09.766800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa826a-ff83-475c-b319-60470ba4bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:09.767697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa826a-ff83-475c-b319-60470ba4bc46 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:09.768048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa826a-ff83-475c-b319-60470ba4bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:09.768393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa826a-ff83-475c-b319-60470ba4bc46 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:09.769335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa826a-ff83-475c-b319-60470ba4bc46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:09.775651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa826a-ff83-475c-b319-60470ba4bc46 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:09.904834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 814/1642] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:09 2022] POST /identity/v3/users => generated 308 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:09.907034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cb6ea-4058-4ea3-a88a-d4c88ba31c9a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:09.924109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cb6ea-4058-4ea3-a88a-d4c88ba31c9a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:09.935183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5b2d3-5e32-4c2b-b4c5-f6c8fb006228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:09.937205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5b2d3-5e32-4c2b-b4c5-f6c8fb006228 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:09.938856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5b2d3-5e32-4c2b-b4c5-f6c8fb006228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:09.939218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc5b2d3-5e32-4c2b-b4c5-f6c8fb006228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:09.940375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5b2d3-5e32-4c2b-b4c5-f6c8fb006228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:09.944940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc5b2d3-5e32-4c2b-b4c5-f6c8fb006228 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:09.986091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 829/1643] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:09.986948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa826a-ff83-475c-b319-60470ba4bc46 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:10.008330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa826a-ff83-475c-b319-60470ba4bc46 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:10.027961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0520e5a-a09e-4fd1-95ae-948a9f50c344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:10.028734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0520e5a-a09e-4fd1-95ae-948a9f50c344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:10.028928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0520e5a-a09e-4fd1-95ae-948a9f50c344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:10.029199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0520e5a-a09e-4fd1-95ae-948a9f50c344 admin admin] PATH_INFO: `/v3/projects/beff5653020e404cbd266552b1a24fd0/users/6b4131f4122d43e29431552981b90292/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:10.057908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:10.064288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0520e5a-a09e-4fd1-95ae-948a9f50c344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beff5653020e404cbd266552b1a24fd0, user_id=6b4131f4122d43e29431552981b90292, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:10.067107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0520e5a-a09e-4fd1-95ae-948a9f50c344 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:10.156596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 815/1644] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:09 2022] PUT /identity/v3/projects/beff5653020e404cbd266552b1a24fd0/users/6b4131f4122d43e29431552981b90292/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:10.160889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5b2d3-5e32-4c2b-b4c5-f6c8fb006228 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:10.175271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc5b2d3-5e32-4c2b-b4c5-f6c8fb006228 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:10.295645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:10.338644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e2d28-70f8-485a-8133-802f7ce4fbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:10.339281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2d28-70f8-485a-8133-802f7ce4fbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:10.339561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2d28-70f8-485a-8133-802f7ce4fbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:10.339825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e2d28-70f8-485a-8133-802f7ce4fbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:10.340500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2d28-70f8-485a-8133-802f7ce4fbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:10.342170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e2d28-70f8-485a-8133-802f7ce4fbe5 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:10.375013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 830/1645] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:10.379483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0520e5a-a09e-4fd1-95ae-948a9f50c344 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:10.401683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0520e5a-a09e-4fd1-95ae-948a9f50c344 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:10.445614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d71094-8a3f-4f0a-8e8f-8895b3e060e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:10.463707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71094-8a3f-4f0a-8e8f-8895b3e060e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:10.465139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71094-8a3f-4f0a-8e8f-8895b3e060e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:10.465417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d71094-8a3f-4f0a-8e8f-8895b3e060e3 admin admin] PATH_INFO: `/v3/projects/beff5653020e404cbd266552b1a24fd0/users/6b4131f4122d43e29431552981b90292/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:10.470533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d71094-8a3f-4f0a-8e8f-8895b3e060e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beff5653020e404cbd266552b1a24fd0, user_id=6b4131f4122d43e29431552981b90292, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:10.479026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d71094-8a3f-4f0a-8e8f-8895b3e060e3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:10.512007 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 816/1646] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:10 2022] PUT /identity/v3/projects/beff5653020e404cbd266552b1a24fd0/users/6b4131f4122d43e29431552981b90292/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:10.544728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220717-9b56-45a5-931d-ebd034fffb73 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:10.545427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220717-9b56-45a5-931d-ebd034fffb73 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:10.545809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f220717-9b56-45a5-931d-ebd034fffb73 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:10.548159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 831/1647] 10.222.0.20 () {54 vars in 962 bytes} [Fri Sep 9 23:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:10.565409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b948797-1312-4d49-afb6-09c4de26a719 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:10.565671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b948797-1312-4d49-afb6-09c4de26a719 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:10.565948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b948797-1312-4d49-afb6-09c4de26a719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:10.676832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7b948797-1312-4d49-afb6-09c4de26a719 None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:10.687376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7b948797-1312-4d49-afb6-09c4de26a719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:10.870711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 817/1648] 10.222.0.20 () {58 vars in 1059 bytes} [Fri Sep 9 23:21:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:10.879594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978859d-a060-4c20-85ab-633a87b945c9 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:10.879882 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978859d-a060-4c20-85ab-633a87b945c9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:10.880246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978859d-a060-4c20-85ab-633a87b945c9 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:10.882141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 832/1649] 10.222.0.20 () {54 vars in 912 bytes} [Fri Sep 9 23:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:10.888500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22162c8f-c49e-49e6-9e2f-0a3b8b388b0b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:10.888938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22162c8f-c49e-49e6-9e2f-0a3b8b388b0b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:10.889349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22162c8f-c49e-49e6-9e2f-0a3b8b388b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:10.912336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-22162c8f-c49e-49e6-9e2f-0a3b8b388b0b None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:10.913841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-22162c8f-c49e-49e6-9e2f-0a3b8b388b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:10.933828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 818/1650] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:21:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:10.946480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978859d-a060-4c20-85ab-633a87b945c9 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:11.058285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978859d-a060-4c20-85ab-633a87b945c9 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:11.064590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a04604-e8e8-4044-b317-75866658c719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:11.065416 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a04604-e8e8-4044-b317-75866658c719 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:11.066058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a04604-e8e8-4044-b317-75866658c719 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:11.066501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a04604-e8e8-4044-b317-75866658c719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:11.087117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a04604-e8e8-4044-b317-75866658c719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:11.089064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a04604-e8e8-4044-b317-75866658c719 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:11.137165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 833/1651] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:12.607666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22162c8f-c49e-49e6-9e2f-0a3b8b388b0b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:12.614899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:12.792004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22162c8f-c49e-49e6-9e2f-0a3b8b388b0b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:12.813476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:12.813476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:12.813476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:12.813476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] PATH_INFO: `/v3/users/989f2baf667d4ae5b1c176f3aa0ac3b4` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:12.814006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] RBAC: Authorizing `identity:delete_user(user_id=989f2baf667d4ae5b1c176f3aa0ac3b4)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:12.814006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: WARNING py.warnings [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:21:12.814006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: warnings.warn(msg) Sep 09 23:21:12.814006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: Sep 09 23:21:12.814798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:13.194933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989f2baf667d4ae5b1c176f3aa0ac3b4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:13.273376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989f2baf667d4ae5b1c176f3aa0ac3b4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:13.274412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989f2baf667d4ae5b1c176f3aa0ac3b4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:13.293596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989f2baf667d4ae5b1c176f3aa0ac3b4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:13.338801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989f2baf667d4ae5b1c176f3aa0ac3b4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:13.351836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 819/1652] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:12 2022] DELETE /identity/v3/users/989f2baf667d4ae5b1c176f3aa0ac3b4 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:13.356311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dada909-9cf4-424c-affa-e997bb47dd52 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:13.357209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dada909-9cf4-424c-affa-e997bb47dd52 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:13.357623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dada909-9cf4-424c-affa-e997bb47dd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:13.397337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4dada909-9cf4-424c-affa-e997bb47dd52 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:13.398698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4dada909-9cf4-424c-affa-e997bb47dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:13.472454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 834/1653] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:13.506206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:13.512200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:13.632708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1dc95-c98f-44ec-8747-1237dd42807e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:13.645018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b141bba-eacd-43f5-ae90-2e763b98da4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:13.646387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b141bba-eacd-43f5-ae90-2e763b98da4d service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:13.649418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b141bba-eacd-43f5-ae90-2e763b98da4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:13.649418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b141bba-eacd-43f5-ae90-2e763b98da4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:13.658323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e94dae-d3f2-45e3-a9a2-2e6fb51cb99e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:13.658873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e94dae-d3f2-45e3-a9a2-2e6fb51cb99e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:13.659294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e94dae-d3f2-45e3-a9a2-2e6fb51cb99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:13.679579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b141bba-eacd-43f5-ae90-2e763b98da4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:13.683158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b141bba-eacd-43f5-ae90-2e763b98da4d service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:13.700663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 820/1654] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:13.712237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c0e94dae-d3f2-45e3-a9a2-2e6fb51cb99e None None] MFA Rules not processed for user `6b4131f4122d43e29431552981b90292`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:13.713728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c0e94dae-d3f2-45e3-a9a2-2e6fb51cb99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:13.908873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 835/1655] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:13 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:15.082621 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64729f24-deaf-4558-993d-af95fe9d9079 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:15.083972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64729f24-deaf-4558-993d-af95fe9d9079 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:15.084267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64729f24-deaf-4558-993d-af95fe9d9079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:15.111166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e94dae-d3f2-45e3-a9a2-2e6fb51cb99e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:15.113733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c0e94dae-d3f2-45e3-a9a2-2e6fb51cb99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:15.157432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e94dae-d3f2-45e3-a9a2-2e6fb51cb99e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:15.172830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:15.172830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:15.172830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:15.172830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] PATH_INFO: `/v3/projects/bef846fcbf2442c1b35a7a98fc782fcf` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:15.173246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-64729f24-deaf-4558-993d-af95fe9d9079 None None] MFA Rules not processed for user `a496b0d69dd24d6980fa7301b7fa2a90`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:15.173349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef846fcbf2442c1b35a7a98fc782fcf)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:15.177200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: WARNING py.warnings [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 09 23:21:15.177200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: warnings.warn(msg) Sep 09 23:21:15.177200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: Sep 09 23:21:15.178435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:15.179974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-64729f24-deaf-4558-993d-af95fe9d9079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:15.212367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:15.237005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:15.362011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 821/1656] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:15.373451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6778e-9f1b-4dcb-8af4-142923eb6552 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:15.373889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6778e-9f1b-4dcb-8af4-142923eb6552 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:15.374275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6778e-9f1b-4dcb-8af4-142923eb6552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:15.427920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-93c6778e-9f1b-4dcb-8af4-142923eb6552 None None] MFA Rules not processed for user `a496b0d69dd24d6980fa7301b7fa2a90`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:15.434955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-93c6778e-9f1b-4dcb-8af4-142923eb6552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:15.491622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef846fcbf2442c1b35a7a98fc782fcf', 'request_id': 'req-5c2cf441-31f9-47ac-884e-55a1b6f23346'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:15.553727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef846fcbf2442c1b35a7a98fc782fcf', 'request_id': 'req-5c2cf441-31f9-47ac-884e-55a1b6f23346'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:15.575739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef846fcbf2442c1b35a7a98fc782fcf', 'request_id': 'req-5c2cf441-31f9-47ac-884e-55a1b6f23346'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:15.578962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 822/1657] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:15.597971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6778e-9f1b-4dcb-8af4-142923eb6552 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:15.600329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-93c6778e-9f1b-4dcb-8af4-142923eb6552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:15.694047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] The token cache is being invalidate because project bef846fcbf2442c1b35a7a98fc782fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:15.696337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:15.698603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:15.703051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 836/1658] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:15 2022] DELETE /identity/v3/projects/bef846fcbf2442c1b35a7a98fc782fcf => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:15.707573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:15.709931 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:15.755066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6778e-9f1b-4dcb-8af4-142923eb6552 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:15.772173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af551c-dfe5-4c05-8d62-90cddbaf3ad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:15.772946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af551c-dfe5-4c05-8d62-90cddbaf3ad3 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:15.773328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af551c-dfe5-4c05-8d62-90cddbaf3ad3 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:15.773702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af551c-dfe5-4c05-8d62-90cddbaf3ad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:15.785015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a7af551c-dfe5-4c05-8d62-90cddbaf3ad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:15.812398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af551c-dfe5-4c05-8d62-90cddbaf3ad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:15.814303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af551c-dfe5-4c05-8d62-90cddbaf3ad3 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:15.838544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 823/1659] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:15.901066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2cf441-31f9-47ac-884e-55a1b6f23346 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:15.912982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:15.913757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:15.914279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:15.914737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] PATH_INFO: `/v3/users/4dd7009df75345da82e7fe63d537ebd7` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:15.917025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd7009df75345da82e7fe63d537ebd7)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:15.919165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:16.191261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd7009df75345da82e7fe63d537ebd7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:16.192478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd7009df75345da82e7fe63d537ebd7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:16.258624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd7009df75345da82e7fe63d537ebd7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:16.280443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf9b76-f7f4-4ef5-b50f-eea7a5e7da18 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:16.282644 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf9b76-f7f4-4ef5-b50f-eea7a5e7da18 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:16.284274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf9b76-f7f4-4ef5-b50f-eea7a5e7da18 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:16.289981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 824/1660] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:21:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:16.290143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd7009df75345da82e7fe63d537ebd7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:16.304891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd7009df75345da82e7fe63d537ebd7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:16.315645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 837/1661] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:15 2022] DELETE /identity/v3/users/4dd7009df75345da82e7fe63d537ebd7 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:16.322372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f840e7-75ab-46b9-8715-f8287d5e5eee None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:16.322936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f840e7-75ab-46b9-8715-f8287d5e5eee None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:16.323559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f840e7-75ab-46b9-8715-f8287d5e5eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:16.364166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-82f840e7-75ab-46b9-8715-f8287d5e5eee None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:16.368710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-82f840e7-75ab-46b9-8715-f8287d5e5eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:16.459354 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:16.519162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 825/1662] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:21:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:16.565074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:16.566659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:16.598562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0412e-fa81-405d-8488-458e502c14a2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:16.636746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:16.637401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:16.639800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:16.643942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:16.680423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:16.686019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:16.693915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dc3f4-10a9-472e-88c2-779c4d8e183d None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:16.700096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dc3f4-10a9-472e-88c2-779c4d8e183d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:16.700599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dc3f4-10a9-472e-88c2-779c4d8e183d None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:16.705649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 826/1663] 10.222.0.20 () {54 vars in 912 bytes} [Fri Sep 9 23:21:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:16.710790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e6fb8-ec1d-4017-a267-7ca0a37d92af None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:16.711588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e6fb8-ec1d-4017-a267-7ca0a37d92af None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:16.712148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e6fb8-ec1d-4017-a267-7ca0a37d92af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:16.744270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:16.752238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 838/1664] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:16.775255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-325e6fb8-ec1d-4017-a267-7ca0a37d92af None None] MFA Rules not processed for user `a9da2d7ef80f4622b58c9041b65df992`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:16.777007 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-325e6fb8-ec1d-4017-a267-7ca0a37d92af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:16.883346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 827/1665] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:21:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:16.963599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:16.965753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:16.989619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6b707-ee0f-4482-a609-b27849068e75 service placement] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:17.002050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:17.002861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:17.003242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:17.005208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:17.017368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:17.019625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:17.030613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 839/1666] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:17.925901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e6fb8-ec1d-4017-a267-7ca0a37d92af None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:17.927369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-325e6fb8-ec1d-4017-a267-7ca0a37d92af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:18.121462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e6fb8-ec1d-4017-a267-7ca0a37d92af None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:18.144105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:18.145473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:18.145955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:18.146883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] PATH_INFO: `/v3/users/42cede68d7fd414e991916c17a262385` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:18.155186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42cede68d7fd414e991916c17a262385)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:18.158053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:18.230277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:18.231991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:18.259705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ef4ce-5b87-4ff2-b9e8-0fedb4c309d5 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:18.265296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:18.267045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:18.267618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:18.268099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] PATH_INFO: `/v3/projects/31b13b9b8e45421484add01a80144412` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:18.269869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b13b9b8e45421484add01a80144412)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:18.272532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:18.455385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b13b9b8e45421484add01a80144412', 'request_id': 'req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.471286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42cede68d7fd414e991916c17a262385'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.621531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b13b9b8e45421484add01a80144412', 'request_id': 'req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.632267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b13b9b8e45421484add01a80144412', 'request_id': 'req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.663030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] The token cache is being invalidate because project 31b13b9b8e45421484add01a80144412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:18.664234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.666082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-28cd53f1-7621-4c8f-ad39-5b01d8a2b47c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.671010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 840/1667] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:18 2022] DELETE /identity/v3/projects/31b13b9b8e45421484add01a80144412 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:18.679146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae93a7c-4136-47d5-8070-aeb8a611d4df None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:18.679587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae93a7c-4136-47d5-8070-aeb8a611d4df None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:18.679997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae93a7c-4136-47d5-8070-aeb8a611d4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:18.750027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3ae93a7c-4136-47d5-8070-aeb8a611d4df None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:18.753291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3ae93a7c-4136-47d5-8070-aeb8a611d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:18.770059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42cede68d7fd414e991916c17a262385'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.771020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42cede68d7fd414e991916c17a262385'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.783175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42cede68d7fd414e991916c17a262385'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.815780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:18.819189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42cede68d7fd414e991916c17a262385'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:18.836852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 828/1668] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:17 2022] DELETE /identity/v3/users/42cede68d7fd414e991916c17a262385 => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:18.836852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:18.838909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:18.995310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 841/1669] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:19.011917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d299dd-2d59-41ca-964a-c413c069fcc7 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:19.012207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d299dd-2d59-41ca-964a-c413c069fcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:19.012488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d299dd-2d59-41ca-964a-c413c069fcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:19.050382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f503f5-a519-467a-bbad-c0474f278041 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:19.063068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af0ff3d-7bc0-4582-a4a0-68a147329778 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:19.064433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0ff3d-7bc0-4582-a4a0-68a147329778 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:19.064433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0ff3d-7bc0-4582-a4a0-68a147329778 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:19.064604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af0ff3d-7bc0-4582-a4a0-68a147329778 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:19.065905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-43d299dd-2d59-41ca-964a-c413c069fcc7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:19.067849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-43d299dd-2d59-41ca-964a-c413c069fcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:19.071698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0af0ff3d-7bc0-4582-a4a0-68a147329778 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:19.113839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0ff3d-7bc0-4582-a4a0-68a147329778 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:19.118178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af0ff3d-7bc0-4582-a4a0-68a147329778 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:19.157564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 829/1670] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:19.161001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eb7ad-0084-48ff-8623-066840935ea6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:19.162991 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eb7ad-0084-48ff-8623-066840935ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:19.163316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3eb7ad-0084-48ff-8623-066840935ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:19.180546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 842/1671] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:19.187115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d299dd-2d59-41ca-964a-c413c069fcc7 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:19.214842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d299dd-2d59-41ca-964a-c413c069fcc7 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:19.232070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:19.233955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:19.234672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:19.235745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:19.236585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-be3eb7ad-0084-48ff-8623-066840935ea6 None None] MFA Rules not processed for user `987d2f6bb1ff4f1fa248c3ff410c8119`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:19.237711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:19.237976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-be3eb7ad-0084-48ff-8623-066840935ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:19.238459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:19.242774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:19.266264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:19.279968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 843/1672] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:19.280898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:19.323275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ac91c-9e25-4e78-b117-5b147d887bbc admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:19.339429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:19.339823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:19.340430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:19.340673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:19.341326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:19.345641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:19.416436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 830/1673] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:19.429226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4754-9da8-47e8-a4f7-2e31a2a47340 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:19.432862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4754-9da8-47e8-a4f7-2e31a2a47340 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:19.433858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c4754-9da8-47e8-a4f7-2e31a2a47340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:19.453960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 844/1674] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:19 2022] POST /identity/v3/projects => generated 401 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:19.453960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:19.455955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:19.507693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ae4c4754-9da8-47e8-a4f7-2e31a2a47340 None None] MFA Rules not processed for user `987d2f6bb1ff4f1fa248c3ff410c8119`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:19.513218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ae4c4754-9da8-47e8-a4f7-2e31a2a47340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:19.563729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d6a17d-bd44-4208-b934-58aea8fa1a3f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:19.582373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:19.583071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:19.583349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:19.583617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] PATH_INFO: `/v3/projects/d611d0dcf35842838e4b6acea505a622` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:19.604826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d611d0dcf35842838e4b6acea505a622)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:19.604826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:19.731405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 831/1675] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:19.732658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c4754-9da8-47e8-a4f7-2e31a2a47340 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:19.759742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c4754-9da8-47e8-a4f7-2e31a2a47340 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:19.766905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:19.767608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:19.767882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:19.768150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:19.768798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:19.770945 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:19.809411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd611d0dcf35842838e4b6acea505a622', 'request_id': 'req-f9bdea4e-7d67-4417-abf6-74c731960258'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:19.886774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 832/1676] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:19 2022] POST /identity/v3/users => generated 301 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:19.886774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:19.893987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:19.938740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd611d0dcf35842838e4b6acea505a622', 'request_id': 'req-f9bdea4e-7d67-4417-abf6-74c731960258'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:19.946098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd611d0dcf35842838e4b6acea505a622', 'request_id': 'req-f9bdea4e-7d67-4417-abf6-74c731960258'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:19.975723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] The token cache is being invalidate because project d611d0dcf35842838e4b6acea505a622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:19.976019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:19.978749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:19.989646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 845/1677] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:19 2022] DELETE /identity/v3/projects/d611d0dcf35842838e4b6acea505a622 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:19.990667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:19.999670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:20.049749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b6ee4-f946-4609-b679-56458cbc6bba admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:20.154961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdea4e-7d67-4417-abf6-74c731960258 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:20.169094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:20.170264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:20.172208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:20.173083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:20.174494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:20.180071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:20.187608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f5c48-c1a3-45a6-8651-72bde2dfee80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:20.188820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f5c48-c1a3-45a6-8651-72bde2dfee80 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:20.189101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f5c48-c1a3-45a6-8651-72bde2dfee80 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:20.189369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f5c48-c1a3-45a6-8651-72bde2dfee80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:20.193683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ee7f5c48-c1a3-45a6-8651-72bde2dfee80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:20.205869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 846/1678] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:20.206942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:20.210317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:20.228625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30630f8-eca6-4664-8a75-82e8fb457bfc admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:20.235034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:20.236233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:20.236535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:20.236796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] PATH_INFO: `/v3/users/6b4131f4122d43e29431552981b90292` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:20.238191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4131f4122d43e29431552981b90292)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:20.243096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:20.311734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f5c48-c1a3-45a6-8651-72bde2dfee80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:20.320081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f5c48-c1a3-45a6-8651-72bde2dfee80 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:20.331261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 833/1679] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:20.335652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1578d9b-4008-4297-ae01-248ff4f49876 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:20.335652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1578d9b-4008-4297-ae01-248ff4f49876 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:20.335652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1578d9b-4008-4297-ae01-248ff4f49876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:20.389377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f1578d9b-4008-4297-ae01-248ff4f49876 None None] MFA Rules not processed for user `f097c4fc270748cc8d41d7bcaba672a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:20.400696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f1578d9b-4008-4297-ae01-248ff4f49876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:20.543178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4131f4122d43e29431552981b90292'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:20.544897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4131f4122d43e29431552981b90292'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:20.576702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 834/1680] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:20.581024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1578d9b-4008-4297-ae01-248ff4f49876 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:20.604701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1578d9b-4008-4297-ae01-248ff4f49876 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:20.622844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:20.623498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:20.623775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:20.624058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:20.624708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:20.626660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:20.650107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4131f4122d43e29431552981b90292'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:20.668896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 835/1681] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:20.671074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:20.671608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4131f4122d43e29431552981b90292'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:20.676554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:20.682870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4131f4122d43e29431552981b90292'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:20.697162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 847/1682] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:20 2022] DELETE /identity/v3/users/6b4131f4122d43e29431552981b90292 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:20.702160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:20.708462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:20.721059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b2a9-f2db-4544-a084-0c5831c55098 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:20.840882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df173246-c876-446c-9c9f-3d2cb76e37bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:20.841818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df173246-c876-446c-9c9f-3d2cb76e37bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:20.842201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df173246-c876-446c-9c9f-3d2cb76e37bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:20.843534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df173246-c876-446c-9c9f-3d2cb76e37bb admin admin] PATH_INFO: `/v3/projects/a8e3141f344a45bdb8a95f86bcc31e36/users/ba7c57a2dc4a441db6b4d890bac6af90/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:20.847099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:20.855638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ee118-cf26-46f0-b5ce-e9ab97b65442 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:20.862633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07a3fa-9471-4b65-9874-a808fd348acb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:20.863305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07a3fa-9471-4b65-9874-a808fd348acb service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:20.863654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07a3fa-9471-4b65-9874-a808fd348acb service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:20.863944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07a3fa-9471-4b65-9874-a808fd348acb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:20.865774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8c07a3fa-9471-4b65-9874-a808fd348acb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:20.884948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df173246-c876-446c-9c9f-3d2cb76e37bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8e3141f344a45bdb8a95f86bcc31e36, user_id=ba7c57a2dc4a441db6b4d890bac6af90, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:20.888201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df173246-c876-446c-9c9f-3d2cb76e37bb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:20.951398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07a3fa-9471-4b65-9874-a808fd348acb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:20.954286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07a3fa-9471-4b65-9874-a808fd348acb service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:20.968468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 848/1683] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:21:20.972824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 836/1684] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:20 2022] PUT /identity/v3/projects/a8e3141f344a45bdb8a95f86bcc31e36/users/ba7c57a2dc4a441db6b4d890bac6af90/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:20.984204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999cbfd-93d1-40c1-b50d-6a3357bf7789 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:20.986171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999cbfd-93d1-40c1-b50d-6a3357bf7789 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:20.986824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999cbfd-93d1-40c1-b50d-6a3357bf7789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:21.055955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0999cbfd-93d1-40c1-b50d-6a3357bf7789 None None] MFA Rules not processed for user `ba7c57a2dc4a441db6b4d890bac6af90`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:21.061496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0999cbfd-93d1-40c1-b50d-6a3357bf7789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:21.201395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 849/1685] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:20 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:21.206880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df173246-c876-446c-9c9f-3d2cb76e37bb admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:21.307920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df173246-c876-446c-9c9f-3d2cb76e37bb admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:21.356792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365271e9-58ca-4a34-bded-7d4b80cd7f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:21.357333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365271e9-58ca-4a34-bded-7d4b80cd7f89 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:21.358108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365271e9-58ca-4a34-bded-7d4b80cd7f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:21.358300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365271e9-58ca-4a34-bded-7d4b80cd7f89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:21.359410 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365271e9-58ca-4a34-bded-7d4b80cd7f89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:21.362070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365271e9-58ca-4a34-bded-7d4b80cd7f89 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:21.495043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999cbfd-93d1-40c1-b50d-6a3357bf7789 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:21.499364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0999cbfd-93d1-40c1-b50d-6a3357bf7789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:21.556629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999cbfd-93d1-40c1-b50d-6a3357bf7789 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:21.579302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 837/1686] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:21 2022] POST /identity/v3/projects => generated 399 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:21.583533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365271e9-58ca-4a34-bded-7d4b80cd7f89 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:21.595721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:21.596371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:21.596643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:21.626529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:21.626529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] PATH_INFO: `/v3/projects/beff5653020e404cbd266552b1a24fd0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:21.629942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=beff5653020e404cbd266552b1a24fd0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:21.632268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:21.798459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365271e9-58ca-4a34-bded-7d4b80cd7f89 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:21.804472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fcfec-b85b-41e7-aaad-842a20373fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:21.805592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fcfec-b85b-41e7-aaad-842a20373fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:21.805950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fcfec-b85b-41e7-aaad-842a20373fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:21.806298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fcfec-b85b-41e7-aaad-842a20373fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:21.807107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fcfec-b85b-41e7-aaad-842a20373fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:21.810403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fcfec-b85b-41e7-aaad-842a20373fd9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:21.932779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 838/1687] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:21 2022] POST /identity/v3/users => generated 306 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:21.940988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fcfec-b85b-41e7-aaad-842a20373fd9 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:21.996189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beff5653020e404cbd266552b1a24fd0', 'request_id': 'req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:22.011528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fcfec-b85b-41e7-aaad-842a20373fd9 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:22.127132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908e88b-60cc-4d23-9fe9-725092bfe509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:22.131772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beff5653020e404cbd266552b1a24fd0', 'request_id': 'req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:22.134249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e88b-60cc-4d23-9fe9-725092bfe509 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:22.137541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e88b-60cc-4d23-9fe9-725092bfe509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:22.137820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908e88b-60cc-4d23-9fe9-725092bfe509 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:22.143956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908e88b-60cc-4d23-9fe9-725092bfe509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:22.143956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908e88b-60cc-4d23-9fe9-725092bfe509 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:22.149637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beff5653020e404cbd266552b1a24fd0', 'request_id': 'req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:22.154747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 839/1688] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:22.159408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908e88b-60cc-4d23-9fe9-725092bfe509 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:22.175644 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908e88b-60cc-4d23-9fe9-725092bfe509 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:22.204543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] The token cache is being invalidate because project beff5653020e404cbd266552b1a24fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:22.206151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:22.210701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:22.211254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-fb8cf961-b694-47e0-a0e3-ce1c275bbf6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:22.220545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 850/1689] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:21 2022] DELETE /identity/v3/projects/beff5653020e404cbd266552b1a24fd0 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:22.220750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:22.220750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:22.220750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] PATH_INFO: `/v3/projects/1e3f97aa7ab14a85b60209d929a2a522/users/5d02aa13e52c4244912afff5088e2f13/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:22.237671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fefec-e0c6-4e26-8d21-da74d4c34a0c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:22.250810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fefec-e0c6-4e26-8d21-da74d4c34a0c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:22.250810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fefec-e0c6-4e26-8d21-da74d4c34a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:22.283252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3f97aa7ab14a85b60209d929a2a522, user_id=5d02aa13e52c4244912afff5088e2f13, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:22.301801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:22.311729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ee4fefec-e0c6-4e26-8d21-da74d4c34a0c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:22.317110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ee4fefec-e0c6-4e26-8d21-da74d4c34a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:22.355609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 851/1690] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:22.374822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f24c2-5180-4f20-b258-40286ccea39b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:22.377005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f24c2-5180-4f20-b258-40286ccea39b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:22.377344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f24c2-5180-4f20-b258-40286ccea39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:22.420555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 840/1691] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:22 2022] PUT /identity/v3/projects/1e3f97aa7ab14a85b60209d929a2a522/users/5d02aa13e52c4244912afff5088e2f13/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:22.421570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:22.427146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:22.559939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2e1f24c2-5180-4f20-b258-40286ccea39b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:22.561195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2e1f24c2-5180-4f20-b258-40286ccea39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:22.658294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:22.773882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d2771-e27d-4a4a-b1cf-86f20ce4bc56 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:22.802790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:22.803844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:22.804765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:22.805301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:22.806235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:22.808212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:22.847844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 841/1692] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:22.849127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:22.855671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:22.858128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 852/1693] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:22.859232 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f24c2-5180-4f20-b258-40286ccea39b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:22.897740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fc7b6-c245-4edf-bbde-45e50e570284 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:22.911280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:22.911738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f24c2-5180-4f20-b258-40286ccea39b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:22.911952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:22.912918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:22.913196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] PATH_INFO: `/v3/users/f29c9543ad85449cacc674d92fba578b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:22.915731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f29c9543ad85449cacc674d92fba578b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:22.933317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:22.940356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d64c35-35c7-430e-91fc-867658c1f970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:22.944003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d64c35-35c7-430e-91fc-867658c1f970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:22.947752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d64c35-35c7-430e-91fc-867658c1f970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:22.948144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d64c35-35c7-430e-91fc-867658c1f970 admin admin] PATH_INFO: `/v3/projects/1e3f97aa7ab14a85b60209d929a2a522/users/5d02aa13e52c4244912afff5088e2f13/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:22.957311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d64c35-35c7-430e-91fc-867658c1f970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3f97aa7ab14a85b60209d929a2a522, user_id=5d02aa13e52c4244912afff5088e2f13, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:22.990233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d64c35-35c7-430e-91fc-867658c1f970 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:23.013673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 853/1694] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:22 2022] PUT /identity/v3/projects/1e3f97aa7ab14a85b60209d929a2a522/users/5d02aa13e52c4244912afff5088e2f13/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:23.015183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d64c35-35c7-430e-91fc-867658c1f970 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:23.192703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d64c35-35c7-430e-91fc-867658c1f970 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:23.207354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f2bc2-afd7-4cfa-843e-714c3ab093a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:23.207988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f2bc2-afd7-4cfa-843e-714c3ab093a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:23.208278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f2bc2-afd7-4cfa-843e-714c3ab093a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:23.208557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f2bc2-afd7-4cfa-843e-714c3ab093a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:23.209214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f2bc2-afd7-4cfa-843e-714c3ab093a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:23.209521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f2bc2-afd7-4cfa-843e-714c3ab093a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:23.211260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f2bc2-afd7-4cfa-843e-714c3ab093a5 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:23.223009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f29c9543ad85449cacc674d92fba578b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:23.253452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 854/1695] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:23.262456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5ca7-4327-4734-b5a6-6a8a160650ec None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:23.262768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5ca7-4327-4734-b5a6-6a8a160650ec None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:23.263028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5ca7-4327-4734-b5a6-6a8a160650ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:23.351603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f29c9543ad85449cacc674d92fba578b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:23.355625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d44a5ca7-4327-4734-b5a6-6a8a160650ec None None] MFA Rules not processed for user `987d2f6bb1ff4f1fa248c3ff410c8119`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:23.355791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f29c9543ad85449cacc674d92fba578b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:23.361613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d44a5ca7-4327-4734-b5a6-6a8a160650ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:23.374721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f29c9543ad85449cacc674d92fba578b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:23.403323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-20cd52e1-9c46-4a5d-b8b6-8ec68c6acec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f29c9543ad85449cacc674d92fba578b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:23.421547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 842/1696] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:22 2022] DELETE /identity/v3/users/f29c9543ad85449cacc674d92fba578b => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:23.429505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a129bf2-9d12-4234-9990-a6a27ba4d088 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:23.430281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a129bf2-9d12-4234-9990-a6a27ba4d088 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:23.431033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a129bf2-9d12-4234-9990-a6a27ba4d088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:23.523940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9a129bf2-9d12-4234-9990-a6a27ba4d088 None None] MFA Rules not processed for user `5d02aa13e52c4244912afff5088e2f13`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:23.526135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9a129bf2-9d12-4234-9990-a6a27ba4d088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:23.541446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 855/1697] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:23.548586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a5ca7-4327-4734-b5a6-6a8a160650ec None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:23.550648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d44a5ca7-4327-4734-b5a6-6a8a160650ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:23.582253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a5ca7-4327-4734-b5a6-6a8a160650ec None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:23.615787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:23.773753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:23.775813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:23.776762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:23.777438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:23.784265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:23.789184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 843/1698] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:23 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:23.791080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:23.801162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439a73d-b553-4002-9b6d-cffd2dab168a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:23.801558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439a73d-b553-4002-9b6d-cffd2dab168a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:23.801977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439a73d-b553-4002-9b6d-cffd2dab168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:23.839114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8439a73d-b553-4002-9b6d-cffd2dab168a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:23.840521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8439a73d-b553-4002-9b6d-cffd2dab168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:23.859075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 844/1699] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:23.862433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439a73d-b553-4002-9b6d-cffd2dab168a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:23.866169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8439a73d-b553-4002-9b6d-cffd2dab168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:23.983045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 856/1700] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:23 2022] POST /identity/v3/projects => generated 355 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:23.984107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:23.991111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:24.029613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439a73d-b553-4002-9b6d-cffd2dab168a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:24.170135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dca2e8-a388-4e17-9490-fc092e1df211 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:24.176503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225afee-2810-4a36-919b-df9590d0c19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:24.177603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225afee-2810-4a36-919b-df9590d0c19d service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.178123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225afee-2810-4a36-919b-df9590d0c19d service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.178640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225afee-2810-4a36-919b-df9590d0c19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.194577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:24.195205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.195715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.196130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.319663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225afee-2810-4a36-919b-df9590d0c19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:24.324384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225afee-2810-4a36-919b-df9590d0c19d service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:24.358775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 857/1701] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:24.359641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225afee-2810-4a36-919b-df9590d0c19d service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:24.373432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:24.378104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:24.401616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225afee-2810-4a36-919b-df9590d0c19d service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:24.406551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 845/1702] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:23 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:24.419123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca5532-7a24-47c4-8a23-424a26ddfdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:24.419963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca5532-7a24-47c4-8a23-424a26ddfdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.420250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca5532-7a24-47c4-8a23-424a26ddfdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.420524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca5532-7a24-47c4-8a23-424a26ddfdb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.421166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca5532-7a24-47c4-8a23-424a26ddfdb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:24.422903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:24.427690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca5532-7a24-47c4-8a23-424a26ddfdb6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:24.440931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:24.482425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a240434-e92f-4dd2-8da1-69188983349f service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:24.488148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:24.488915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.489295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.489639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] PATH_INFO: `/v3/users/c25910b8d0554a7bbac3ed1450bb62bc` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.490937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25910b8d0554a7bbac3ed1450bb62bc)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:24.492986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:24.635844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 858/1703] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:24 2022] POST /identity/v3/users => generated 278 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:24.635844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7243898-3a02-4513-812a-056312474d46 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.635844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7243898-3a02-4513-812a-056312474d46 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.635844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7243898-3a02-4513-812a-056312474d46 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.635844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 859/1704] 10.222.0.20 () {54 vars in 912 bytes} [Fri Sep 9 23:21:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:24.643241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7243898-3a02-4513-812a-056312474d46 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:24.695974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7243898-3a02-4513-812a-056312474d46 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:24.713092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce9b33-ec51-4b56-a1cd-e309276446b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:24.715445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9b33-ec51-4b56-a1cd-e309276446b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.718829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9b33-ec51-4b56-a1cd-e309276446b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.725173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce9b33-ec51-4b56-a1cd-e309276446b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.725173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce9b33-ec51-4b56-a1cd-e309276446b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:24.725173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce9b33-ec51-4b56-a1cd-e309276446b2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:24.756094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 860/1705] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:24.757773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81990fc4-1ab1-43f6-b599-33be6bc666ea None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.758061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81990fc4-1ab1-43f6-b599-33be6bc666ea None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.758610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81990fc4-1ab1-43f6-b599-33be6bc666ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.799969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-81990fc4-1ab1-43f6-b599-33be6bc666ea None None] MFA Rules not processed for user `a9da2d7ef80f4622b58c9041b65df992`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:24.801261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-81990fc4-1ab1-43f6-b599-33be6bc666ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:24.820364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 861/1706] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:21:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:24.821249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81990fc4-1ab1-43f6-b599-33be6bc666ea None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:24.823199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25910b8d0554a7bbac3ed1450bb62bc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:24.846050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81990fc4-1ab1-43f6-b599-33be6bc666ea None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:24.851709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:24.852581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.853343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.853877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.854938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:24.856746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:24.872483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 862/1707] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:24.875489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:24.876524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:24.900056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:24.906559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ca179-4454-4616-ba87-cf1db7e82ea6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:24.914410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f9169-9c1a-4165-9262-47b64199da0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:24.915806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9169-9c1a-4165-9262-47b64199da0b service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:24.916086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9169-9c1a-4165-9262-47b64199da0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:24.917581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f9169-9c1a-4165-9262-47b64199da0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:24.926965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25910b8d0554a7bbac3ed1450bb62bc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:24.928819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25910b8d0554a7bbac3ed1450bb62bc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:24.944638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25910b8d0554a7bbac3ed1450bb62bc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:24.949974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f9169-9c1a-4165-9262-47b64199da0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:24.951439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f9169-9c1a-4165-9262-47b64199da0b service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:24.974575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25910b8d0554a7bbac3ed1450bb62bc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:24.979184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 863/1708] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:24.987011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f9169-9c1a-4165-9262-47b64199da0b service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:24.994212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 846/1709] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:24 2022] DELETE /identity/v3/users/c25910b8d0554a7bbac3ed1450bb62bc => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:24.995713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:24.997450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:25.029189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f9169-9c1a-4165-9262-47b64199da0b service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:25.164073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69012def-4424-43ad-a216-1d421b89c67e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:25.173638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61617d-dddc-4c7f-a103-648d7bb0272f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:25.174638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61617d-dddc-4c7f-a103-648d7bb0272f service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:25.176072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61617d-dddc-4c7f-a103-648d7bb0272f service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:25.182265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61617d-dddc-4c7f-a103-648d7bb0272f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:25.184076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9e61617d-dddc-4c7f-a103-648d7bb0272f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:25.191819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa542e-f443-4ecf-bd70-898ed21357cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:25.192514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa542e-f443-4ecf-bd70-898ed21357cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:25.192793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa542e-f443-4ecf-bd70-898ed21357cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:25.193065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa542e-f443-4ecf-bd70-898ed21357cd admin admin] PATH_INFO: `/v3/projects/e6d856552078444d9c2382a1f29c5b3b/users/87bda7c87902448693ae2169631db9fd/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:25.238093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa542e-f443-4ecf-bd70-898ed21357cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d856552078444d9c2382a1f29c5b3b, user_id=87bda7c87902448693ae2169631db9fd, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:25.245135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa542e-f443-4ecf-bd70-898ed21357cd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:25.318148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61617d-dddc-4c7f-a103-648d7bb0272f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:25.319778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61617d-dddc-4c7f-a103-648d7bb0272f service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:25.330517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 847/1710] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:24 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:25.335292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a0c1d-4423-4903-9f1e-77f908fc4e67 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:25.335609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a0c1d-4423-4903-9f1e-77f908fc4e67 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:25.335844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a0c1d-4423-4903-9f1e-77f908fc4e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:25.356627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 864/1711] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:24 2022] PUT /identity/v3/projects/e6d856552078444d9c2382a1f29c5b3b/users/87bda7c87902448693ae2169631db9fd/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:25.359010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60ebca-a75d-4e3c-ad69-749ad0c44edc None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:25.359010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60ebca-a75d-4e3c-ad69-749ad0c44edc None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:25.359010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60ebca-a75d-4e3c-ad69-749ad0c44edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:25.382020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6e8a0c1d-4423-4903-9f1e-77f908fc4e67 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:25.387717 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6e8a0c1d-4423-4903-9f1e-77f908fc4e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:25.503647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6b60ebca-a75d-4e3c-ad69-749ad0c44edc None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:25.504908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6b60ebca-a75d-4e3c-ad69-749ad0c44edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:25.572851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 848/1712] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:25.576786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680b28-2851-43b4-89e1-6faf6cb10171 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:25.578395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680b28-2851-43b4-89e1-6faf6cb10171 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:25.579758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52680b28-2851-43b4-89e1-6faf6cb10171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:25.664034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-52680b28-2851-43b4-89e1-6faf6cb10171 None None] MFA Rules not processed for user `87bda7c87902448693ae2169631db9fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:25.665770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-52680b28-2851-43b4-89e1-6faf6cb10171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:25.683279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 865/1713] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:21:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:25.690265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60ebca-a75d-4e3c-ad69-749ad0c44edc None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:25.693122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6b60ebca-a75d-4e3c-ad69-749ad0c44edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:25.788105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 849/1714] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:25.788853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52680b28-2851-43b4-89e1-6faf6cb10171 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:25.790399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-52680b28-2851-43b4-89e1-6faf6cb10171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:25.819138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52680b28-2851-43b4-89e1-6faf6cb10171 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:25.832441 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:25.833223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:25.833718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:25.834058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:25.835434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:25.838080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60ebca-a75d-4e3c-ad69-749ad0c44edc None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:25.838229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:25.843955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a99b2a-7ca8-4ca1-a87b-c6d5da7443bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:25.848167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a99b2a-7ca8-4ca1-a87b-c6d5da7443bd service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:25.848167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a99b2a-7ca8-4ca1-a87b-c6d5da7443bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:25.848167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a99b2a-7ca8-4ca1-a87b-c6d5da7443bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:25.890319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a99b2a-7ca8-4ca1-a87b-c6d5da7443bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:25.892749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a99b2a-7ca8-4ca1-a87b-c6d5da7443bd service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:25.908677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 866/1715] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:25.917936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:26.044876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 850/1716] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:25 2022] POST /identity/v3/projects => generated 357 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:26.048769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a99b2a-7ca8-4ca1-a87b-c6d5da7443bd service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:26.064015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a99b2a-7ca8-4ca1-a87b-c6d5da7443bd service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:26.163883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:26.166084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:26.252013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1a96-095f-44b4-b6cd-eb4f4215e76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:26.255231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1a96-095f-44b4-b6cd-eb4f4215e76a admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:26.255231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1a96-095f-44b4-b6cd-eb4f4215e76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:26.256793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae1a96-095f-44b4-b6cd-eb4f4215e76a admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:26.259330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1a96-095f-44b4-b6cd-eb4f4215e76a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:26.264707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae1a96-095f-44b4-b6cd-eb4f4215e76a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:26.356202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dafb736-1deb-41d9-9b75-f2edf75f54a1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:26.356202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:26.356202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:26.356202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:26.357041 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:26.442395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:26.444882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:26.463421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 851/1717] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 23:21:26.465053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:26.466869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:26.488896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 867/1718] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:26 2022] POST /identity/v3/users => generated 285 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:26.512062 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805aea4-3e54-4f36-a20e-70e7fa012836 service swift] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:26.522022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1a96-095f-44b4-b6cd-eb4f4215e76a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:26.554029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:26.557987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:26.560558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:26.560843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] PATH_INFO: `/v3/projects/d9dc686533584d00b764ff90330a6d0b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:26.566999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9dc686533584d00b764ff90330a6d0b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:26.573754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae1a96-095f-44b4-b6cd-eb4f4215e76a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:26.574165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:26.595726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026577b-c8b9-4792-9871-6284db7d043c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:26.597910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026577b-c8b9-4792-9871-6284db7d043c admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:26.599223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026577b-c8b9-4792-9871-6284db7d043c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:26.600491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026577b-c8b9-4792-9871-6284db7d043c admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:26.602879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026577b-c8b9-4792-9871-6284db7d043c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:26.608708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c026577b-c8b9-4792-9871-6284db7d043c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:26.673983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 868/1719] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:26.673983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026577b-c8b9-4792-9871-6284db7d043c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:26.714997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026577b-c8b9-4792-9871-6284db7d043c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:26.731253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec29d9-30e6-4ba0-8cd8-b77b375d2051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:26.732461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec29d9-30e6-4ba0-8cd8-b77b375d2051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:26.733087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec29d9-30e6-4ba0-8cd8-b77b375d2051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:26.733559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec29d9-30e6-4ba0-8cd8-b77b375d2051 admin admin] PATH_INFO: `/v3/projects/a0101fde4d0c4ddb8a7a70f09e313618/users/3a4d25eedb3e4b328954295d00c94184/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:26.767578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec29d9-30e6-4ba0-8cd8-b77b375d2051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0101fde4d0c4ddb8a7a70f09e313618, user_id=3a4d25eedb3e4b328954295d00c94184, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:26.770646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec29d9-30e6-4ba0-8cd8-b77b375d2051 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:26.852372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 869/1720] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:26 2022] PUT /identity/v3/projects/a0101fde4d0c4ddb8a7a70f09e313618/users/3a4d25eedb3e4b328954295d00c94184/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:26.857104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec29d9-30e6-4ba0-8cd8-b77b375d2051 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:26.895048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9dc686533584d00b764ff90330a6d0b', 'request_id': 'req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:27.010608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9dc686533584d00b764ff90330a6d0b', 'request_id': 'req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:27.054473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9dc686533584d00b764ff90330a6d0b', 'request_id': 'req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:27.069487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec29d9-30e6-4ba0-8cd8-b77b375d2051 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:27.099332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] The token cache is being invalidate because project d9dc686533584d00b764ff90330a6d0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:27.099943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:27.101059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ab78e852-898c-4f5f-a16d-a4b6a445dfa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:27.108352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 852/1721] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:26 2022] DELETE /identity/v3/projects/d9dc686533584d00b764ff90330a6d0b => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:27.110289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96cc6-a5d9-4fb9-bbbd-99075b48c0f3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:27.114039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96cc6-a5d9-4fb9-bbbd-99075b48c0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:27.114536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96cc6-a5d9-4fb9-bbbd-99075b48c0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:27.172098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-72e96cc6-a5d9-4fb9-bbbd-99075b48c0f3 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:27.175781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-72e96cc6-a5d9-4fb9-bbbd-99075b48c0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:27.264713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:27.266641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:27.268969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:27.269572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:27.271905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:27.273604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:27.331844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 870/1722] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:27.333527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:27.335454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:27.367924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 853/1723] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:27.384854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5f77-cf5e-4ff5-8526-bfe72c2a240c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:27.384854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5f77-cf5e-4ff5-8526-bfe72c2a240c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:27.384854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5f77-cf5e-4ff5-8526-bfe72c2a240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:27.386077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf37cc0-b6bf-41d7-acae-f29c9be1977a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:27.388865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe88eaa-f360-4730-885f-91b0b632fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:27.389659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe88eaa-f360-4730-885f-91b0b632fb4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:27.389943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe88eaa-f360-4730-885f-91b0b632fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:27.390259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe88eaa-f360-4730-885f-91b0b632fb4f admin admin] PATH_INFO: `/v3/projects/a0101fde4d0c4ddb8a7a70f09e313618/users/3a4d25eedb3e4b328954295d00c94184/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:27.400381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe88eaa-f360-4730-885f-91b0b632fb4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0101fde4d0c4ddb8a7a70f09e313618, user_id=3a4d25eedb3e4b328954295d00c94184, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:27.405027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe88eaa-f360-4730-885f-91b0b632fb4f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:27.418279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d12e5f77-cf5e-4ff5-8526-bfe72c2a240c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:27.426699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d12e5f77-cf5e-4ff5-8526-bfe72c2a240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:27.434539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 871/1724] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:27 2022] PUT /identity/v3/projects/a0101fde4d0c4ddb8a7a70f09e313618/users/3a4d25eedb3e4b328954295d00c94184/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:27.447683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e8a1-98da-43de-aa97-5fdb39848f9a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:27.448183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e8a1-98da-43de-aa97-5fdb39848f9a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:27.448567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079e8a1-98da-43de-aa97-5fdb39848f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:27.567358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-9079e8a1-98da-43de-aa97-5fdb39848f9a None None] MFA Rules not processed for user `3a4d25eedb3e4b328954295d00c94184`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:27.586124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9079e8a1-98da-43de-aa97-5fdb39848f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:27.591919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 854/1725] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:27.597538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e5f77-cf5e-4ff5-8526-bfe72c2a240c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:27.632934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e5f77-cf5e-4ff5-8526-bfe72c2a240c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:27.653326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:27.654431 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:27.654779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:27.655140 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:27.664064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:27.664392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:27.675841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:27.728901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 855/1726] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:27.733356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:27.812241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722343bf-927e-45a4-afc7-7319c5c6aad4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:27.857310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:27.863401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:27.870034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:27.870408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 872/1727] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:27.870877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:27.871701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:27.873722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:27.900834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079e8a1-98da-43de-aa97-5fdb39848f9a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:27.904443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:27.906603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9079e8a1-98da-43de-aa97-5fdb39848f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:27.945784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079e8a1-98da-43de-aa97-5fdb39848f9a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:27.982031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:27.983361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:27.983361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:27.986590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] PATH_INFO: `/v3/users/613682a6b93b4a60aaf25a7c72995fe1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:27.989834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=613682a6b93b4a60aaf25a7c72995fe1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:27.991836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:28.007497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 856/1728] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:27 2022] POST /identity/v3/projects => generated 371 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:28.007970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:28.011554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:28.164226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08444d-3f8a-487b-9d0d-2e16eba935fc admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:28.177411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19e9d5-0d6f-4b69-9fc7-cf167470f4de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:28.177411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19e9d5-0d6f-4b69-9fc7-cf167470f4de service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:28.177411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19e9d5-0d6f-4b69-9fc7-cf167470f4de service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:28.177411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19e9d5-0d6f-4b69-9fc7-cf167470f4de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:28.424481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '613682a6b93b4a60aaf25a7c72995fe1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:28.426720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '613682a6b93b4a60aaf25a7c72995fe1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:28.451404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19e9d5-0d6f-4b69-9fc7-cf167470f4de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:28.454501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b19e9d5-0d6f-4b69-9fc7-cf167470f4de service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:28.484481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 857/1729] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:28.486425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19e9d5-0d6f-4b69-9fc7-cf167470f4de service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:28.526112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '613682a6b93b4a60aaf25a7c72995fe1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:28.564943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '613682a6b93b4a60aaf25a7c72995fe1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:28.585888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '613682a6b93b4a60aaf25a7c72995fe1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:28.602225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 873/1730] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:27 2022] DELETE /identity/v3/users/613682a6b93b4a60aaf25a7c72995fe1 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:28.620067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:28.621489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:28.649430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b19e9d5-0d6f-4b69-9fc7-cf167470f4de service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:28.711192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:28.825959 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b85b2d-3a3a-42e8-8f72-5b6a6aee5df2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:28.837652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:28.839020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:28.839456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:28.839933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] PATH_INFO: `/v3/projects/191adf2320154d4297c2dc5f6cf23de8` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:28.844400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f858-0216-4c12-a7f0-e87aabbb9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:28.845351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f858-0216-4c12-a7f0-e87aabbb9ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:28.845702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f858-0216-4c12-a7f0-e87aabbb9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:28.846055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f858-0216-4c12-a7f0-e87aabbb9ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:28.846827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7f858-0216-4c12-a7f0-e87aabbb9ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:28.848997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7f858-0216-4c12-a7f0-e87aabbb9ca5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:28.877917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=191adf2320154d4297c2dc5f6cf23de8)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:28.900454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:29.020272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 858/1731] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:28 2022] POST /identity/v3/users => generated 286 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:29.020272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faca4-73ec-4765-8fa1-75a67bcefca0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:29.020272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faca4-73ec-4765-8fa1-75a67bcefca0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:29.020272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446faca4-73ec-4765-8fa1-75a67bcefca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:29.084809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-446faca4-73ec-4765-8fa1-75a67bcefca0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:29.108906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-446faca4-73ec-4765-8fa1-75a67bcefca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:29.154728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '191adf2320154d4297c2dc5f6cf23de8', 'request_id': 'req-978fe893-871f-4eb3-b111-7091e09b9bc2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:29.260652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '191adf2320154d4297c2dc5f6cf23de8', 'request_id': 'req-978fe893-871f-4eb3-b111-7091e09b9bc2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:29.273315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '191adf2320154d4297c2dc5f6cf23de8', 'request_id': 'req-978fe893-871f-4eb3-b111-7091e09b9bc2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:29.292083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] The token cache is being invalidate because project 191adf2320154d4297c2dc5f6cf23de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:29.292596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:29.294669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:29.308271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 874/1732] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:28 2022] DELETE /identity/v3/projects/191adf2320154d4297c2dc5f6cf23de8 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:29.308271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:29.311893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:29.398519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 859/1733] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:29.402306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446faca4-73ec-4765-8fa1-75a67bcefca0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:29.409418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-446faca4-73ec-4765-8fa1-75a67bcefca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:29.433102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446faca4-73ec-4765-8fa1-75a67bcefca0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:29.445071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927a015-2d07-4873-88d5-f26e09bb07c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:29.445905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927a015-2d07-4873-88d5-f26e09bb07c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:29.446332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927a015-2d07-4873-88d5-f26e09bb07c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:29.446790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927a015-2d07-4873-88d5-f26e09bb07c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:29.447588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927a015-2d07-4873-88d5-f26e09bb07c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:29.449478 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927a015-2d07-4873-88d5-f26e09bb07c1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:29.463356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 860/1734] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:29.464951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b272e8-802f-4c4e-a203-31c4a5913612 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:29.465389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b272e8-802f-4c4e-a203-31c4a5913612 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:29.465976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b272e8-802f-4c4e-a203-31c4a5913612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:29.471031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fe893-871f-4eb3-b111-7091e09b9bc2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:29.476437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:29.477149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:29.477507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:29.477784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:29.480841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:29.491895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-47b272e8-802f-4c4e-a203-31c4a5913612 None None] MFA Rules not processed for user `f097c4fc270748cc8d41d7bcaba672a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:29.494575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-47b272e8-802f-4c4e-a203-31c4a5913612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:29.583057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:29.584670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:29.615121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 861/1735] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:29.615252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 875/1736] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:29.615338 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662b04e-e1f1-480a-80a6-442182f744a0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:29.615338 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662b04e-e1f1-480a-80a6-442182f744a0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:29.615484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:29.615559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662b04e-e1f1-480a-80a6-442182f744a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:29.615656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:29.653474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d7e38-d452-463f-ad00-ee810656b18a service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:29.661950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:29.662956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:29.663423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:29.663894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] PATH_INFO: `/v3/users/aa421da9eff24e96ab0e606a5d20d121` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:29.665315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa421da9eff24e96ab0e606a5d20d121)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:29.667591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:29.684315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5662b04e-e1f1-480a-80a6-442182f744a0 None None] MFA Rules not processed for user `a9da2d7ef80f4622b58c9041b65df992`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:29.684315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5662b04e-e1f1-480a-80a6-442182f744a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:29.808934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 862/1737] 10.222.0.42 () {58 vars in 1015 bytes} [Fri Sep 9 23:21:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:29.810597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662b04e-e1f1-480a-80a6-442182f744a0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:29.812035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5662b04e-e1f1-480a-80a6-442182f744a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:29.825535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662b04e-e1f1-480a-80a6-442182f744a0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:29.831095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694d8ea-f238-4250-b5ef-ba349b4f35b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:29.831768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694d8ea-f238-4250-b5ef-ba349b4f35b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:29.832099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694d8ea-f238-4250-b5ef-ba349b4f35b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:29.832375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694d8ea-f238-4250-b5ef-ba349b4f35b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:29.864157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694d8ea-f238-4250-b5ef-ba349b4f35b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:29.868678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694d8ea-f238-4250-b5ef-ba349b4f35b8 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:29.910297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 863/1738] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:29.912180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694d8ea-f238-4250-b5ef-ba349b4f35b8 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:29.947948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694d8ea-f238-4250-b5ef-ba349b4f35b8 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:29.960777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:29.962762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:29.966261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:29.966261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:29.966261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:29.967482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:30.009818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa421da9eff24e96ab0e606a5d20d121'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:30.011442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa421da9eff24e96ab0e606a5d20d121'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:30.019879 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 864/1739] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:30.020888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:30.027244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:30.054991 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b708f0-f511-48c2-8d0a-0bdc9d2424c1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:30.073976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:30.074375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:30.074660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:30.074932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] PATH_INFO: `/v3/projects/19de8edc106444bca0b21c4564a84a60/users/8714e6cfbf54441fa0eedf66f6fa5e2c/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:30.105177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19de8edc106444bca0b21c4564a84a60, user_id=8714e6cfbf54441fa0eedf66f6fa5e2c, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:30.108622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:30.120760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa421da9eff24e96ab0e606a5d20d121'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:30.171439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa421da9eff24e96ab0e606a5d20d121'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:30.172536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa421da9eff24e96ab0e606a5d20d121'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:30.194064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 876/1740] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:29 2022] DELETE /identity/v3/users/aa421da9eff24e96ab0e606a5d20d121 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:30.223406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:30.224964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:30.374541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 865/1741] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:30 2022] PUT /identity/v3/projects/19de8edc106444bca0b21c4564a84a60/users/8714e6cfbf54441fa0eedf66f6fa5e2c/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:30.376692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:30.389611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:30.612303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fe493-f31c-49dd-927e-d2cec2ccad9d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:30.623349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79107b13-fe27-496c-b039-61c97728c051 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:30.639750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:30.640402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:30.640881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:30.642942 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:30.645481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa7429-8276-48cc-baf3-206231442966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:30.646288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7429-8276-48cc-baf3-206231442966 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:30.646562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7429-8276-48cc-baf3-206231442966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:30.646674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:30.646838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa7429-8276-48cc-baf3-206231442966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:30.650768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ceaa7429-8276-48cc-baf3-206231442966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:30.687698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa7429-8276-48cc-baf3-206231442966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:30.687698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa7429-8276-48cc-baf3-206231442966 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:30.693596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:30.697014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 877/1742] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:30.698714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72de1-c3ca-46be-99a8-fa326bc88aa4 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:30.699294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72de1-c3ca-46be-99a8-fa326bc88aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:30.699779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a72de1-c3ca-46be-99a8-fa326bc88aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:30.710487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:30.738160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 866/1743] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:30.746135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-41a72de1-c3ca-46be-99a8-fa326bc88aa4 None None] MFA Rules not processed for user `8714e6cfbf54441fa0eedf66f6fa5e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:30.748729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-41a72de1-c3ca-46be-99a8-fa326bc88aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:30.850864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 878/1744] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:30.855872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:30.908323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:31.082841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ab608-28d1-4375-8a27-c22d5a245da1 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:31.100668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf35a-bff1-4c08-961e-cea4840877d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:31.101309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf35a-bff1-4c08-961e-cea4840877d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:31.101638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf35a-bff1-4c08-961e-cea4840877d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:31.101866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf35a-bff1-4c08-961e-cea4840877d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:31.102535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccf35a-bff1-4c08-961e-cea4840877d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:31.104755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dccf35a-bff1-4c08-961e-cea4840877d7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:31.247133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 867/1745] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:30 2022] POST /identity/v3/projects => generated 371 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:31.251523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a72de1-c3ca-46be-99a8-fa326bc88aa4 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:31.276371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a72de1-c3ca-46be-99a8-fa326bc88aa4 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:31.366885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a0776-a226-4280-86b7-0a10ac374e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:31.367720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a0776-a226-4280-86b7-0a10ac374e6b admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:31.368235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a0776-a226-4280-86b7-0a10ac374e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:31.381318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a0776-a226-4280-86b7-0a10ac374e6b admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:31.382326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a0776-a226-4280-86b7-0a10ac374e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:31.386843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a0776-a226-4280-86b7-0a10ac374e6b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:31.668433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 879/1746] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:31 2022] POST /identity/v3/users => generated 292 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:31.677070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf35a-bff1-4c08-961e-cea4840877d7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:31.692288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf35a-bff1-4c08-961e-cea4840877d7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:31.713013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ef895-3ec7-40a3-88af-e4f08989694a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:31.714490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ef895-3ec7-40a3-88af-e4f08989694a admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:31.714959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ef895-3ec7-40a3-88af-e4f08989694a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:31.715415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ef895-3ec7-40a3-88af-e4f08989694a admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:31.718284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ef895-3ec7-40a3-88af-e4f08989694a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:31.720850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ef895-3ec7-40a3-88af-e4f08989694a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:31.764223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 868/1747] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:31.767963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a0776-a226-4280-86b7-0a10ac374e6b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:31.790533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a0776-a226-4280-86b7-0a10ac374e6b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:31.801274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:31.808680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:31.810878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:31.810878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] PATH_INFO: `/v3/projects/3a3e6822ec7f4a4c8673bf6b8143a47d/users/616e5969950c4a78bce5ce691ed9577b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:31.842671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3e6822ec7f4a4c8673bf6b8143a47d, user_id=616e5969950c4a78bce5ce691ed9577b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:31.849125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:31.973245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 880/1748] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:31 2022] PUT /identity/v3/projects/3a3e6822ec7f4a4c8673bf6b8143a47d/users/616e5969950c4a78bce5ce691ed9577b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:31.979384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ef895-3ec7-40a3-88af-e4f08989694a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:32.021605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ef895-3ec7-40a3-88af-e4f08989694a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:32.088590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:32.090669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:32.383917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006783c-79f0-4b8c-967d-f15fdcce89ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:32.383917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006783c-79f0-4b8c-967d-f15fdcce89ca admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:32.383917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006783c-79f0-4b8c-967d-f15fdcce89ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:32.383917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006783c-79f0-4b8c-967d-f15fdcce89ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:32.385585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006783c-79f0-4b8c-967d-f15fdcce89ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:32.385585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006783c-79f0-4b8c-967d-f15fdcce89ca admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:32.385585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 869/1749] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:32.388565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99ab19-3a40-445b-9d68-d57103c783ef admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:32.388565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:32.388565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:32.388565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:32.389565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] PATH_INFO: `/v3/projects/7e6ad4ba8591498d824b56ac52cb2725` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:32.389565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6ad4ba8591498d824b56ac52cb2725)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:32.389565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:32.391706 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006783c-79f0-4b8c-967d-f15fdcce89ca admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:32.424985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006783c-79f0-4b8c-967d-f15fdcce89ca admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:32.435036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d41c3e-dd54-4376-a50f-47f1ffef6a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:32.439729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41c3e-dd54-4376-a50f-47f1ffef6a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:32.440175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41c3e-dd54-4376-a50f-47f1ffef6a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:32.442261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d41c3e-dd54-4376-a50f-47f1ffef6a96 admin admin] PATH_INFO: `/v3/projects/3a3e6822ec7f4a4c8673bf6b8143a47d/users/616e5969950c4a78bce5ce691ed9577b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:32.447056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d41c3e-dd54-4376-a50f-47f1ffef6a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3e6822ec7f4a4c8673bf6b8143a47d, user_id=616e5969950c4a78bce5ce691ed9577b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:32.455965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d41c3e-dd54-4376-a50f-47f1ffef6a96 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:32.482402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 870/1750] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:32 2022] PUT /identity/v3/projects/3a3e6822ec7f4a4c8673bf6b8143a47d/users/616e5969950c4a78bce5ce691ed9577b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:32.490906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382e12-f0f5-47c4-bb00-ed87566bed1c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:32.491203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382e12-f0f5-47c4-bb00-ed87566bed1c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:32.491686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5382e12-f0f5-47c4-bb00-ed87566bed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:32.513963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:32.571082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6ad4ba8591498d824b56ac52cb2725', 'request_id': 'req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:32.576032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a5382e12-f0f5-47c4-bb00-ed87566bed1c None None] MFA Rules not processed for user `616e5969950c4a78bce5ce691ed9577b`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:32.579169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a5382e12-f0f5-47c4-bb00-ed87566bed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:32.788138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 871/1751] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:32.825271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5382e12-f0f5-47c4-bb00-ed87566bed1c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:32.833701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a5382e12-f0f5-47c4-bb00-ed87566bed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:32.855808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6ad4ba8591498d824b56ac52cb2725', 'request_id': 'req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:32.871722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6ad4ba8591498d824b56ac52cb2725', 'request_id': 'req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:32.895237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] The token cache is being invalidate because project 7e6ad4ba8591498d824b56ac52cb2725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:32.895464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:32.898238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:32.915141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 881/1752] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:32 2022] DELETE /identity/v3/projects/7e6ad4ba8591498d824b56ac52cb2725 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:32.916178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:32.923332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:32.981278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5382e12-f0f5-47c4-bb00-ed87566bed1c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:32.983576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ed259-32b4-4732-8325-34842f10dd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:32.984344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed259-32b4-4732-8325-34842f10dd3b service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:32.984654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed259-32b4-4732-8325-34842f10dd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:32.984955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ed259-32b4-4732-8325-34842f10dd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:32.986811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bd3ed259-32b4-4732-8325-34842f10dd3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:33.003725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ed259-32b4-4732-8325-34842f10dd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:33.005246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ed259-32b4-4732-8325-34842f10dd3b service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:33.029028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 872/1753] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:21:33.033321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f455e-287a-415c-8446-676dd9fe07ff None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:33.033321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f455e-287a-415c-8446-676dd9fe07ff None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:33.035821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f455e-287a-415c-8446-676dd9fe07ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:33.110947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d83f455e-287a-415c-8446-676dd9fe07ff None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:33.110947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d83f455e-287a-415c-8446-676dd9fe07ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:33.183399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed321d-94ac-4d61-b3a2-c1b3144cb8d6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:33.239040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:33.239698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:33.239991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:33.240249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] PATH_INFO: `/v3/projects/5819879d09b8412e8127cd03ca90529a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:33.252440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5819879d09b8412e8127cd03ca90529a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:33.260717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:33.360346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 873/1754] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:33.360346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f455e-287a-415c-8446-676dd9fe07ff None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:33.365884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d83f455e-287a-415c-8446-676dd9fe07ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:33.379549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f455e-287a-415c-8446-676dd9fe07ff None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:33.384138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:33.392517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:33.393376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:33.393807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:33.394230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] PATH_INFO: `/v3/users/ba7c57a2dc4a441db6b4d890bac6af90` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:33.395811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7c57a2dc4a441db6b4d890bac6af90)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:33.398014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:33.732696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5819879d09b8412e8127cd03ca90529a', 'request_id': 'req-e5ede4f3-f984-490f-8c52-95c181fea677'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:33.737409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7c57a2dc4a441db6b4d890bac6af90'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:33.867111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7c57a2dc4a441db6b4d890bac6af90'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:33.871404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7c57a2dc4a441db6b4d890bac6af90'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:33.900286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7c57a2dc4a441db6b4d890bac6af90'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:33.918975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a4aab6da-1dbc-42f5-8f42-ecefbd30b8b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7c57a2dc4a441db6b4d890bac6af90'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:33.939201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 874/1755] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:33 2022] DELETE /identity/v3/users/ba7c57a2dc4a441db6b4d890bac6af90 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:33.939201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef0e5-bbac-40a1-87af-f3aceb49d366 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:33.939201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef0e5-bbac-40a1-87af-f3aceb49d366 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:33.939201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ef0e5-bbac-40a1-87af-f3aceb49d366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:33.963008 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-052ef0e5-bbac-40a1-87af-f3aceb49d366 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:33.964718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-052ef0e5-bbac-40a1-87af-f3aceb49d366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:33.967165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5819879d09b8412e8127cd03ca90529a', 'request_id': 'req-e5ede4f3-f984-490f-8c52-95c181fea677'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:33.976843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5819879d09b8412e8127cd03ca90529a', 'request_id': 'req-e5ede4f3-f984-490f-8c52-95c181fea677'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:34.091387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] The token cache is being invalidate because project 5819879d09b8412e8127cd03ca90529a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:34.094645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:34.099142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e5ede4f3-f984-490f-8c52-95c181fea677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:34.107861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 882/1756] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:32 2022] DELETE /identity/v3/projects/5819879d09b8412e8127cd03ca90529a => generated 0 bytes in 1186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:34.107861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d55e6-0a93-4e45-879f-b2b248fa2885 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.107861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d55e6-0a93-4e45-879f-b2b248fa2885 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.107861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d55e6-0a93-4e45-879f-b2b248fa2885 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.123355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 883/1757] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:21:34 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:34.129840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e53b-0707-4475-ad6f-cdc06cf39f72 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.132154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e53b-0707-4475-ad6f-cdc06cf39f72 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.133289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14e53b-0707-4475-ad6f-cdc06cf39f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.157970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 875/1758] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:34.161310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87694772-60cf-48e8-8674-8a66820ec8cb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.161592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87694772-60cf-48e8-8674-8a66820ec8cb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.161868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87694772-60cf-48e8-8674-8a66820ec8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.201700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-fc14e53b-0707-4475-ad6f-cdc06cf39f72 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:34.208321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fc14e53b-0707-4475-ad6f-cdc06cf39f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:34.217210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-87694772-60cf-48e8-8674-8a66820ec8cb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:34.220301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-87694772-60cf-48e8-8674-8a66820ec8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:34.222375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 884/1759] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:34.224151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b3d8a-740a-402a-9e13-f39518bb789b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.224435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b3d8a-740a-402a-9e13-f39518bb789b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.224726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b3d8a-740a-402a-9e13-f39518bb789b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.247877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 876/1760] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:34.250984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87694772-60cf-48e8-8674-8a66820ec8cb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:34.278575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87694772-60cf-48e8-8674-8a66820ec8cb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:34.287267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:34.288192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.288672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.289199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.290696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:34.291828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:34.293938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:34.298959 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c06b3d8a-740a-402a-9e13-f39518bb789b None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:34.300234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c06b3d8a-740a-402a-9e13-f39518bb789b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:34.360493 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 877/1761] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:34.365747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:34.372049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:34.475104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 885/1762] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:21:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:34.483292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46563f-e4de-4d3f-a013-ae27c00ff688 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.483596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46563f-e4de-4d3f-a013-ae27c00ff688 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.483874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46563f-e4de-4d3f-a013-ae27c00ff688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.545660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ef46563f-e4de-4d3f-a013-ae27c00ff688 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:34.549427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ef46563f-e4de-4d3f-a013-ae27c00ff688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:34.574366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 886/1763] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:34.574366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46563f-e4de-4d3f-a013-ae27c00ff688 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:34.654979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46563f-e4de-4d3f-a013-ae27c00ff688 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:34.675973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fa559-14e2-4099-a44b-85b0da429be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:34.676602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa559-14e2-4099-a44b-85b0da429be0 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.676877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa559-14e2-4099-a44b-85b0da429be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.677148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fa559-14e2-4099-a44b-85b0da429be0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.677788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fa559-14e2-4099-a44b-85b0da429be0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:34.679939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fa559-14e2-4099-a44b-85b0da429be0 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:34.845571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85127c-b692-4cc7-939f-3d486e66760c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:34.845571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a1bd-73eb-4fe6-9231-bf9d30bbd33a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:34.845571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a1bd-73eb-4fe6-9231-bf9d30bbd33a service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.845571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a1bd-73eb-4fe6-9231-bf9d30bbd33a service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.846568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a1bd-73eb-4fe6-9231-bf9d30bbd33a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.846568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5a1bd-73eb-4fe6-9231-bf9d30bbd33a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:34.846568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5a1bd-73eb-4fe6-9231-bf9d30bbd33a service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:34.846568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 878/1764] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:34.846568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a1bd-73eb-4fe6-9231-bf9d30bbd33a service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:34.846568 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a1bd-73eb-4fe6-9231-bf9d30bbd33a service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:34.860657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf51ce-cc9b-4d27-898a-89b25d38a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:34.862049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf51ce-cc9b-4d27-898a-89b25d38a330 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.862584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf51ce-cc9b-4d27-898a-89b25d38a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.863467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf51ce-cc9b-4d27-898a-89b25d38a330 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.864333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf51ce-cc9b-4d27-898a-89b25d38a330 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:34.865353 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf51ce-cc9b-4d27-898a-89b25d38a330 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:34.868191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf51ce-cc9b-4d27-898a-89b25d38a330 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:34.906988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 879/1765] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:34.908810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc147d-d129-424a-9887-1631c68e29df None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.910627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc147d-d129-424a-9887-1631c68e29df None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.911137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc147d-d129-424a-9887-1631c68e29df None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:34.915408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 880/1766] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:21:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:21:34.916430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc147d-d129-424a-9887-1631c68e29df None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:34.920642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c5bc147d-d129-424a-9887-1631c68e29df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:34.963144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc147d-d129-424a-9887-1631c68e29df None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:34.972492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952db4aa-8b11-4d24-8c2d-9b58f4da967f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:34.973069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db4aa-8b11-4d24-8c2d-9b58f4da967f service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:34.973681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db4aa-8b11-4d24-8c2d-9b58f4da967f service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:34.977044 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952db4aa-8b11-4d24-8c2d-9b58f4da967f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:35.002220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952db4aa-8b11-4d24-8c2d-9b58f4da967f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:35.004123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952db4aa-8b11-4d24-8c2d-9b58f4da967f service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:35.031406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 887/1767] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:34 2022] POST /identity/v3/projects => generated 381 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:35.033489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fa559-14e2-4099-a44b-85b0da429be0 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:35.041804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 881/1768] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:35.071231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d7038-d2bc-4b2f-81b7-945e91afcd7b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:35.071231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d7038-d2bc-4b2f-81b7-945e91afcd7b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:35.071231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d7038-d2bc-4b2f-81b7-945e91afcd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:35.171488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7d1d7038-d2bc-4b2f-81b7-945e91afcd7b None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:35.173075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7d1d7038-d2bc-4b2f-81b7-945e91afcd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:35.264679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fa559-14e2-4099-a44b-85b0da429be0 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:35.278261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:35.278261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:35.278261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:35.278261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:35.279002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:35.280673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:35.419952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 882/1769] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:21:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:35.424335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d7038-d2bc-4b2f-81b7-945e91afcd7b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:35.440958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d7038-d2bc-4b2f-81b7-945e91afcd7b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:35.445966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f19b15-b3eb-485b-a64b-f6a0e7d8fe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:35.447000 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f19b15-b3eb-485b-a64b-f6a0e7d8fe99 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:35.447393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f19b15-b3eb-485b-a64b-f6a0e7d8fe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:35.447768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f19b15-b3eb-485b-a64b-f6a0e7d8fe99 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:35.460974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f19b15-b3eb-485b-a64b-f6a0e7d8fe99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:35.460974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f19b15-b3eb-485b-a64b-f6a0e7d8fe99 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:35.497664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 888/1770] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:35 2022] POST /identity/v3/projects => generated 405 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:35.498599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:35.503404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:35.684197 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c27e8-4e4f-47a9-8216-957e9ca3f5bc admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:35.684197 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:35.684197 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:35.684197 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:35.684808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] PATH_INFO: `/v3/users/8714e6cfbf54441fa0eedf66f6fa5e2c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:35.684808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8714e6cfbf54441fa0eedf66f6fa5e2c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:35.684808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:35.795473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 883/1771] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:35 2022] POST /identity/v3/users => generated 291 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:35.795473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f19b15-b3eb-485b-a64b-f6a0e7d8fe99 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:35.810877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f19b15-b3eb-485b-a64b-f6a0e7d8fe99 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:35.899430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:35.941002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d10bec-6390-484b-b69f-57703887e883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:35.941002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d10bec-6390-484b-b69f-57703887e883 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:35.941002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d10bec-6390-484b-b69f-57703887e883 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:35.941002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d10bec-6390-484b-b69f-57703887e883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:35.941656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d10bec-6390-484b-b69f-57703887e883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:35.941656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d10bec-6390-484b-b69f-57703887e883 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:35.962180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 884/1772] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:35.966990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d10bec-6390-484b-b69f-57703887e883 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:36.069703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d10bec-6390-484b-b69f-57703887e883 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:36.081506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:36.084069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.084403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.088273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:36.088273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:36.091433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:36.177863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8714e6cfbf54441fa0eedf66f6fa5e2c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:36.179290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8714e6cfbf54441fa0eedf66f6fa5e2c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:36.232712 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 885/1773] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:35 2022] POST /identity/v3/users => generated 303 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:36.235579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:36.241488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:36.264492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950da2ac-883a-4dff-a1ca-cf9eb133c2ee admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:36.272577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:36.274118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.274875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.275396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:36.276354 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:36.288961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:36.291773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8714e6cfbf54441fa0eedf66f6fa5e2c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:36.319779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 886/1774] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:36.319779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:36.324870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8714e6cfbf54441fa0eedf66f6fa5e2c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:36.327703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:36.351644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8714e6cfbf54441fa0eedf66f6fa5e2c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:36.358760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec0eee-4787-4f77-ba81-9170f283a821 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:36.364711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53253a91-d9ec-4b3c-94cc-409ca9c05479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:36.365773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53253a91-d9ec-4b3c-94cc-409ca9c05479 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.366104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53253a91-d9ec-4b3c-94cc-409ca9c05479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.366466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53253a91-d9ec-4b3c-94cc-409ca9c05479 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:36.367289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53253a91-d9ec-4b3c-94cc-409ca9c05479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:36.368992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53253a91-d9ec-4b3c-94cc-409ca9c05479 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:36.372642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 889/1775] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:35 2022] DELETE /identity/v3/users/8714e6cfbf54441fa0eedf66f6fa5e2c => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:36.377308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:36.382958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 887/1776] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:36.384444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb4d0-4141-4a83-adbf-22ca4aa13b85 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.384714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb4d0-4141-4a83-adbf-22ca4aa13b85 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.385032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fb4d0-4141-4a83-adbf-22ca4aa13b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:36.431533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:36.474141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-027fb4d0-4141-4a83-adbf-22ca4aa13b85 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:36.475483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-027fb4d0-4141-4a83-adbf-22ca4aa13b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:36.578476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353aa88-5d88-415f-8cee-18f5b9ac6cb0 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:36.586389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd696ad-8ee6-4874-bfb1-5a904232b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:36.587096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd696ad-8ee6-4874-bfb1-5a904232b96f admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.587501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd696ad-8ee6-4874-bfb1-5a904232b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.587660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd696ad-8ee6-4874-bfb1-5a904232b96f admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:36.588408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd696ad-8ee6-4874-bfb1-5a904232b96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:36.590722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd696ad-8ee6-4874-bfb1-5a904232b96f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:36.611547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 890/1777] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:36.612731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd696ad-8ee6-4874-bfb1-5a904232b96f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:36.626748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd696ad-8ee6-4874-bfb1-5a904232b96f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:36.634048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5603c0-4832-4e48-b344-e7947b161563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:36.635665 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5603c0-4832-4e48-b344-e7947b161563 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.637144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5603c0-4832-4e48-b344-e7947b161563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.637492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5603c0-4832-4e48-b344-e7947b161563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:36.638378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5603c0-4832-4e48-b344-e7947b161563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:36.643877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5603c0-4832-4e48-b344-e7947b161563 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:36.654728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 888/1778] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:36.656929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fb4d0-4141-4a83-adbf-22ca4aa13b85 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:36.672949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 891/1779] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:36.679051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5603c0-4832-4e48-b344-e7947b161563 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:36.681356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fb4d0-4141-4a83-adbf-22ca4aa13b85 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:36.699776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489f80b-4ad0-4921-8a61-f0fbeb28295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:36.700596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f80b-4ad0-4921-8a61-f0fbeb28295b admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.701029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f80b-4ad0-4921-8a61-f0fbeb28295b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.701364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489f80b-4ad0-4921-8a61-f0fbeb28295b admin admin] PATH_INFO: `/v3/projects/73901ec50eb6483fbcc980ced4eed624/users/50492499974b4cca87eb680ee3a048b6/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:36.704405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5603c0-4832-4e48-b344-e7947b161563 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:36.799459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:36.799459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.799459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.799459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] PATH_INFO: `/v3/projects/6a635c7199ed43b2847cefa75cb1631b/users/343e2af74c434d0f834d29a5fc9092a9/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:36.800496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489f80b-4ad0-4921-8a61-f0fbeb28295b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73901ec50eb6483fbcc980ced4eed624, user_id=50492499974b4cca87eb680ee3a048b6, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:36.800496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489f80b-4ad0-4921-8a61-f0fbeb28295b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:36.801022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a635c7199ed43b2847cefa75cb1631b, user_id=343e2af74c434d0f834d29a5fc9092a9, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:36.806165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:36.958830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 892/1780] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:36 2022] PUT /identity/v3/projects/6a635c7199ed43b2847cefa75cb1631b/users/343e2af74c434d0f834d29a5fc9092a9/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:36.960024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:36.970416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 889/1781] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:36 2022] PUT /identity/v3/projects/73901ec50eb6483fbcc980ced4eed624/users/50492499974b4cca87eb680ee3a048b6/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:36.978700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826291eb-76a7-4c2c-a1c6-2a7523960001 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:36.979095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:36.980705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826291eb-76a7-4c2c-a1c6-2a7523960001 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:36.980996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826291eb-76a7-4c2c-a1c6-2a7523960001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:37.088045 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-826291eb-76a7-4c2c-a1c6-2a7523960001 None None] MFA Rules not processed for user `343e2af74c434d0f834d29a5fc9092a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:37.089415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-826291eb-76a7-4c2c-a1c6-2a7523960001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:37.226528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac4895-41e8-4843-b445-ef14574295e8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:37.226528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:37.226528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:37.226528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:37.228110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:37.228767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 890/1782] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:36 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:37.228767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb138-f816-4db9-96bc-8836ecf9a7d6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:37.228767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb138-f816-4db9-96bc-8836ecf9a7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:37.228767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb138-f816-4db9-96bc-8836ecf9a7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:37.301583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2c7cb138-f816-4db9-96bc-8836ecf9a7d6 None None] MFA Rules not processed for user `50492499974b4cca87eb680ee3a048b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:37.301772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2c7cb138-f816-4db9-96bc-8836ecf9a7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:37.379407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:37.381479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:37.398327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 893/1783] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:37.399601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:37.401382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:37.425344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb3a23-a7fb-49c1-afcf-7e2929161cdf service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:37.433700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:37.434779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:37.435725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:37.436395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] PATH_INFO: `/v3/projects/a8e3141f344a45bdb8a95f86bcc31e36` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:37.439284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8e3141f344a45bdb8a95f86bcc31e36)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:37.450053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:37.621917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 891/1784] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:37.623055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cb138-f816-4db9-96bc-8836ecf9a7d6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:37.645256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cb138-f816-4db9-96bc-8836ecf9a7d6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:37.658334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4136-3740-473f-b6df-bb2dd153eb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:37.659287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4136-3740-473f-b6df-bb2dd153eb3c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:37.659743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4136-3740-473f-b6df-bb2dd153eb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:37.660192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4136-3740-473f-b6df-bb2dd153eb3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:37.661038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb4136-3740-473f-b6df-bb2dd153eb3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:37.663895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb4136-3740-473f-b6df-bb2dd153eb3c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:37.731724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8e3141f344a45bdb8a95f86bcc31e36', 'request_id': 'req-2c719a56-6099-42d8-938d-273ffcada958'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:37.795797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8e3141f344a45bdb8a95f86bcc31e36', 'request_id': 'req-2c719a56-6099-42d8-938d-273ffcada958'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:37.808644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8e3141f344a45bdb8a95f86bcc31e36', 'request_id': 'req-2c719a56-6099-42d8-938d-273ffcada958'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:37.816276 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 892/1785] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:37 2022] POST /identity/v3/projects => generated 403 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:37.817759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4136-3740-473f-b6df-bb2dd153eb3c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:37.854748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] The token cache is being invalidate because project a8e3141f344a45bdb8a95f86bcc31e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:37.855077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:37.857704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:37.864172 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 894/1786] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:37 2022] DELETE /identity/v3/projects/a8e3141f344a45bdb8a95f86bcc31e36 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:37.865048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:37.867360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:37.984988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c719a56-6099-42d8-938d-273ffcada958 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:38.029711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4136-3740-473f-b6df-bb2dd153eb3c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:38.042227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:38.043809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:38.044165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:38.045233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:38.046004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:38.051952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:38.166844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:38.176336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:38.176853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:38.177293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:38.178139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:38.181687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:38.413985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 893/1787] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:37 2022] POST /identity/v3/projects => generated 381 bytes in 597 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:38.415313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:38.427261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:38.496668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 895/1788] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:37 2022] POST /identity/v3/users => generated 308 bytes in 632 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:38.497491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:38.499114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:38.525046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79404209-ede3-4aed-a301-083cde3b940d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:38.530863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:38.573184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988ea4d-f544-4b7b-97ec-54ae867e0f75 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:38.586916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:38.587739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:38.588019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:38.588288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] PATH_INFO: `/v3/users/5d02aa13e52c4244912afff5088e2f13` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:38.591910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d02aa13e52c4244912afff5088e2f13)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:38.594760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:38.672585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a918f3f-58c0-4aac-8da2-d28b34033559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:38.673565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a918f3f-58c0-4aac-8da2-d28b34033559 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:38.673918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a918f3f-58c0-4aac-8da2-d28b34033559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:38.674260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a918f3f-58c0-4aac-8da2-d28b34033559 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:38.675382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a918f3f-58c0-4aac-8da2-d28b34033559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:38.677786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a918f3f-58c0-4aac-8da2-d28b34033559 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:38.932967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 896/1789] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:38 2022] POST /identity/v3/users => generated 297 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:38.932967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a918f3f-58c0-4aac-8da2-d28b34033559 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:38.968576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a918f3f-58c0-4aac-8da2-d28b34033559 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:38.975579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56810b2e-54be-48a0-b19f-82064993bd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:38.976237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810b2e-54be-48a0-b19f-82064993bd4a admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:38.976577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810b2e-54be-48a0-b19f-82064993bd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:38.976835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56810b2e-54be-48a0-b19f-82064993bd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:38.977481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56810b2e-54be-48a0-b19f-82064993bd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:38.986291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56810b2e-54be-48a0-b19f-82064993bd4a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:39.001684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 897/1790] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:39.002725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56810b2e-54be-48a0-b19f-82064993bd4a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:39.017994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56810b2e-54be-48a0-b19f-82064993bd4a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:39.026066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224ae3b-13bc-4c8c-ac02-3d8548d0dc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:39.032395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224ae3b-13bc-4c8c-ac02-3d8548d0dc53 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:39.032395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224ae3b-13bc-4c8c-ac02-3d8548d0dc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:39.032395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224ae3b-13bc-4c8c-ac02-3d8548d0dc53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:39.032395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224ae3b-13bc-4c8c-ac02-3d8548d0dc53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:39.032395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224ae3b-13bc-4c8c-ac02-3d8548d0dc53 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:39.048019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 898/1791] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:39.049614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224ae3b-13bc-4c8c-ac02-3d8548d0dc53 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:39.071415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224ae3b-13bc-4c8c-ac02-3d8548d0dc53 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:39.088230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428b566-5443-4bab-a1cc-ec0957771482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:39.089028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428b566-5443-4bab-a1cc-ec0957771482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:39.089390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428b566-5443-4bab-a1cc-ec0957771482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:39.089716 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428b566-5443-4bab-a1cc-ec0957771482 admin admin] PATH_INFO: `/v3/projects/266ff0f2a34049d288124ce9d545dd7a/users/6f7725beb38a4bac952870541a6fe5a6/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:39.126313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428b566-5443-4bab-a1cc-ec0957771482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266ff0f2a34049d288124ce9d545dd7a, user_id=6f7725beb38a4bac952870541a6fe5a6, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:39.126313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6428b566-5443-4bab-a1cc-ec0957771482 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:39.145012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d02aa13e52c4244912afff5088e2f13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.255020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 899/1792] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:39 2022] PUT /identity/v3/projects/266ff0f2a34049d288124ce9d545dd7a/users/6f7725beb38a4bac952870541a6fe5a6/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:39.255883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428b566-5443-4bab-a1cc-ec0957771482 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:39.302994 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:39.304869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d02aa13e52c4244912afff5088e2f13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.306849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d02aa13e52c4244912afff5088e2f13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.319700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d02aa13e52c4244912afff5088e2f13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.351479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d02aa13e52c4244912afff5088e2f13'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.362225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 894/1793] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:38 2022] DELETE /identity/v3/users/5d02aa13e52c4244912afff5088e2f13 => generated 0 bytes in 947 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:39.363585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6428b566-5443-4bab-a1cc-ec0957771482 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:39.363948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:39.365795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:39.463922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:39.464878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:39.466561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:39.467861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] PATH_INFO: `/v3/projects/c76ada4473804b96abce6c05b23bcdad/users/27d2f1ae66a446818c0397504bda0056/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:39.500012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76ada4473804b96abce6c05b23bcdad, user_id=27d2f1ae66a446818c0397504bda0056, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:39.505322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:39.517469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0fe20-fcfe-4371-84ef-0d74101d762f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:39.522879 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:39.524255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:39.524701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:39.525119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] PATH_INFO: `/v3/projects/19de8edc106444bca0b21c4564a84a60` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:39.527149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] RBAC: Authorizing `identity:delete_project(project_id=19de8edc106444bca0b21c4564a84a60)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:39.535593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:39.764919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 900/1794] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:39 2022] PUT /identity/v3/projects/c76ada4473804b96abce6c05b23bcdad/users/27d2f1ae66a446818c0397504bda0056/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:39.764919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:39.767388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:39.797785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19de8edc106444bca0b21c4564a84a60', 'request_id': 'req-b2c76394-2f12-4cab-a72b-46e23901900d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.850766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d09a87-9b62-43cb-b026-356e194d0293 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:39.891971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19de8edc106444bca0b21c4564a84a60', 'request_id': 'req-b2c76394-2f12-4cab-a72b-46e23901900d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.899182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19de8edc106444bca0b21c4564a84a60', 'request_id': 'req-b2c76394-2f12-4cab-a72b-46e23901900d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.916189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] The token cache is being invalidate because project 19de8edc106444bca0b21c4564a84a60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:39.916439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.917196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:39.919698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 895/1795] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:39 2022] DELETE /identity/v3/projects/19de8edc106444bca0b21c4564a84a60 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:39.920555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:39.922900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:39.926462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:39.927406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:39.928118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:39.928118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:39.928597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:39.930198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:39.943528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c76394-2f12-4cab-a72b-46e23901900d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:39.949620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:39.950389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:39.950670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:39.950936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:39.951648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:39.954150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:39.957313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 901/1796] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:39.957313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:39.959202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:39.970258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 896/1797] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:39.971231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:39.973012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:39.973449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1113b79-9baf-4f0c-a9f6-6baf045af0a8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:39.978168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:39.979099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:39.979378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:39.979650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] PATH_INFO: `/v3/users/616e5969950c4a78bce5ce691ed9577b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:39.980794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616e5969950c4a78bce5ce691ed9577b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:39.982755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:39.986841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29adaae8-3ddc-4490-98a6-f9220c605274 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:39.992021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c1355-b9ca-4405-a056-a65c23143300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:39.992777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c1355-b9ca-4405-a056-a65c23143300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:39.993257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c1355-b9ca-4405-a056-a65c23143300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:39.993779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c1355-b9ca-4405-a056-a65c23143300 admin admin] PATH_INFO: `/v3/projects/266ff0f2a34049d288124ce9d545dd7a/users/6f7725beb38a4bac952870541a6fe5a6/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:39.995695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c1355-b9ca-4405-a056-a65c23143300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266ff0f2a34049d288124ce9d545dd7a, user_id=6f7725beb38a4bac952870541a6fe5a6, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:39.999013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c1355-b9ca-4405-a056-a65c23143300 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:40.010146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 897/1798] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:39 2022] PUT /identity/v3/projects/266ff0f2a34049d288124ce9d545dd7a/users/6f7725beb38a4bac952870541a6fe5a6/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:40.011208 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c1355-b9ca-4405-a056-a65c23143300 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:40.074959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c1355-b9ca-4405-a056-a65c23143300 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:40.080397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc74af-4492-4297-b324-0b0f2f2161bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:40.081057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc74af-4492-4297-b324-0b0f2f2161bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.081330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc74af-4492-4297-b324-0b0f2f2161bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.081604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc74af-4492-4297-b324-0b0f2f2161bc admin admin] PATH_INFO: `/v3/projects/c76ada4473804b96abce6c05b23bcdad/users/27d2f1ae66a446818c0397504bda0056/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.084596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc74af-4492-4297-b324-0b0f2f2161bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76ada4473804b96abce6c05b23bcdad, user_id=27d2f1ae66a446818c0397504bda0056, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:40.086141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc74af-4492-4297-b324-0b0f2f2161bc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:40.104984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 898/1799] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:40 2022] PUT /identity/v3/projects/c76ada4473804b96abce6c05b23bcdad/users/27d2f1ae66a446818c0397504bda0056/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:40.114517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3e781-5725-4cc4-b742-190ca86a8bab None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.114517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3e781-5725-4cc4-b742-190ca86a8bab None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.114517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3e781-5725-4cc4-b742-190ca86a8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.139523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-17f3e781-5725-4cc4-b742-190ca86a8bab None None] MFA Rules not processed for user `6f7725beb38a4bac952870541a6fe5a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:40.140801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-17f3e781-5725-4cc4-b742-190ca86a8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.206660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616e5969950c4a78bce5ce691ed9577b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.208186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616e5969950c4a78bce5ce691ed9577b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.233346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 899/1800] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:40 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:40.233346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3e781-5725-4cc4-b742-190ca86a8bab None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:40.234382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-17f3e781-5725-4cc4-b742-190ca86a8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.297930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616e5969950c4a78bce5ce691ed9577b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.301069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3e781-5725-4cc4-b742-190ca86a8bab None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:40.306268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:40.307382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.307850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.308312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] PATH_INFO: `/v3/projects/1e3f97aa7ab14a85b60209d929a2a522` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.310496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3f97aa7ab14a85b60209d929a2a522)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:40.313604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616e5969950c4a78bce5ce691ed9577b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.314080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:40.323642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9ea5e748-992c-4ec9-bc74-612db3f79692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616e5969950c4a78bce5ce691ed9577b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.330013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 902/1801] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:39 2022] DELETE /identity/v3/users/616e5969950c4a78bce5ce691ed9577b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:40.331418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ad847-0986-4bbe-957f-678930022599 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.331710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ad847-0986-4bbe-957f-678930022599 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.331991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ad847-0986-4bbe-957f-678930022599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.363915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f65ad847-0986-4bbe-957f-678930022599 None None] MFA Rules not processed for user `27d2f1ae66a446818c0397504bda0056`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:40.365144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f65ad847-0986-4bbe-957f-678930022599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.436870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3f97aa7ab14a85b60209d929a2a522', 'request_id': 'req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.445960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 903/1802] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:40 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:40.446841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ad847-0986-4bbe-957f-678930022599 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:40.448393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f65ad847-0986-4bbe-957f-678930022599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.508427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ad847-0986-4bbe-957f-678930022599 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:40.514910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3f97aa7ab14a85b60209d929a2a522', 'request_id': 'req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.522239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3f97aa7ab14a85b60209d929a2a522', 'request_id': 'req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.537291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] The token cache is being invalidate because project 1e3f97aa7ab14a85b60209d929a2a522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:40.537592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.538284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:40.540263 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 900/1803] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:40 2022] DELETE /identity/v3/projects/1e3f97aa7ab14a85b60209d929a2a522 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:40.541052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:40.542404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.577804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b54b1-3ab0-48b9-863c-76f78cb3a113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:40.579051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b54b1-3ab0-48b9-863c-76f78cb3a113 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.579519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b54b1-3ab0-48b9-863c-76f78cb3a113 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.579786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b54b1-3ab0-48b9-863c-76f78cb3a113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.581875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9b1b54b1-3ab0-48b9-863c-76f78cb3a113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.632118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fb972-d37f-4b8e-93a1-f06db727fdf2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:40.652711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8f98b-d412-49ac-af42-4f28a9579bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:40.654120 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f98b-d412-49ac-af42-4f28a9579bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.655211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f98b-d412-49ac-af42-4f28a9579bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.655593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8f98b-d412-49ac-af42-4f28a9579bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.657989 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-26e8f98b-d412-49ac-af42-4f28a9579bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.665680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b54b1-3ab0-48b9-863c-76f78cb3a113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:40.667729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b54b1-3ab0-48b9-863c-76f78cb3a113 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:40.681479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 904/1804] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:40 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:40.683474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8078-8949-4024-96e6-ba643d11e0ae None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.683765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8078-8949-4024-96e6-ba643d11e0ae None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.684173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f8078-8949-4024-96e6-ba643d11e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.712408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0a8f8078-8949-4024-96e6-ba643d11e0ae None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:40.714185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0a8f8078-8949-4024-96e6-ba643d11e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.733220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:40.759197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8f98b-d412-49ac-af42-4f28a9579bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:40.761334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8f98b-d412-49ac-af42-4f28a9579bb1 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:40.778114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 901/1805] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:40.829162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 905/1806] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:40.841468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e85319-3725-45db-b9bf-2a99a820e765 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.841468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e85319-3725-45db-b9bf-2a99a820e765 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.841468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e85319-3725-45db-b9bf-2a99a820e765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.858695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c8e85319-3725-45db-b9bf-2a99a820e765 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:40.860543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c8e85319-3725-45db-b9bf-2a99a820e765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.873521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 902/1807] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:40.874142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f8078-8949-4024-96e6-ba643d11e0ae None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:40.885587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f8078-8949-4024-96e6-ba643d11e0ae None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:40.890459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:40.891804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.892109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.892381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.893035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:40.893336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:40.894941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:40.910117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 906/1808] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:40.917224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e85319-3725-45db-b9bf-2a99a820e765 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:40.928538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e85319-3725-45db-b9bf-2a99a820e765 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:40.934467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:40.935197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.935595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.935967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.937023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:40.942558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:40.947969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:40.949867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:40.971302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ec48c-bd80-4b90-acec-c65a054ad195 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:40.979718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:40.980417 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:40.980744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:40.981016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] PATH_INFO: `/v3/projects/3a3e6822ec7f4a4c8673bf6b8143a47d` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:40.982145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a3e6822ec7f4a4c8673bf6b8143a47d)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:40.987679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.059223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 903/1809] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:40 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:41.059223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.059223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.106116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a3e6822ec7f4a4c8673bf6b8143a47d', 'request_id': 'req-05725db9-4207-4752-adec-9d253449d63c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.150522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129366d-dc57-49e1-a0ca-bb8b8e5b2b4c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:41.206159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a3e6822ec7f4a4c8673bf6b8143a47d', 'request_id': 'req-05725db9-4207-4752-adec-9d253449d63c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.209946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:41.210548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.210820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.211276 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] PATH_INFO: `/v3/users/343e2af74c434d0f834d29a5fc9092a9` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.212401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=343e2af74c434d0f834d29a5fc9092a9)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:41.214409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.224689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a3e6822ec7f4a4c8673bf6b8143a47d', 'request_id': 'req-05725db9-4207-4752-adec-9d253449d63c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.228042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] The token cache is being invalidate because project 3a3e6822ec7f4a4c8673bf6b8143a47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:41.228315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.229247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.235590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 907/1810] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:40 2022] DELETE /identity/v3/projects/3a3e6822ec7f4a4c8673bf6b8143a47d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:41.237047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.238320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.256794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05725db9-4207-4752-adec-9d253449d63c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:41.262214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4eaa21-bb9e-457e-b6ba-0592bce753b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:41.262921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4eaa21-bb9e-457e-b6ba-0592bce753b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.263194 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4eaa21-bb9e-457e-b6ba-0592bce753b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.263461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4eaa21-bb9e-457e-b6ba-0592bce753b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.264448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4eaa21-bb9e-457e-b6ba-0592bce753b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:41.266712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4eaa21-bb9e-457e-b6ba-0592bce753b6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.366376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 908/1811] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:41 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:41.367947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559dee5-8805-4bb6-a579-b28700b791da None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.367947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559dee5-8805-4bb6-a579-b28700b791da None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.368222 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559dee5-8805-4bb6-a579-b28700b791da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.388243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6559dee5-8805-4bb6-a579-b28700b791da None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:41.389446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6559dee5-8805-4bb6-a579-b28700b791da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.397565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 909/1812] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:41.398356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559dee5-8805-4bb6-a579-b28700b791da None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.409375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559dee5-8805-4bb6-a579-b28700b791da None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:41.415215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aa6e4-c955-4f88-a71a-35171e2ee4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:41.417956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aa6e4-c955-4f88-a71a-35171e2ee4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.417956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aa6e4-c955-4f88-a71a-35171e2ee4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.417956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aa6e4-c955-4f88-a71a-35171e2ee4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.417956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526aa6e4-c955-4f88-a71a-35171e2ee4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:41.418878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526aa6e4-c955-4f88-a71a-35171e2ee4f8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.433620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 910/1813] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:41.433620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a34fa-2b75-489b-a974-5c9f68e966bf None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.433620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a34fa-2b75-489b-a974-5c9f68e966bf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.433620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a34fa-2b75-489b-a974-5c9f68e966bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.459703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-166a34fa-2b75-489b-a974-5c9f68e966bf None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:41.462655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-166a34fa-2b75-489b-a974-5c9f68e966bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.486445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343e2af74c434d0f834d29a5fc9092a9'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.486940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 911/1814] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:41.486940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a34fa-2b75-489b-a974-5c9f68e966bf None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.486940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a34fa-2b75-489b-a974-5c9f68e966bf None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:41.491292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:41.491885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.492164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.492433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.493096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:41.494724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.505066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 912/1815] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:41.505856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.507519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.520528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab195c4-de19-4807-a43f-c458b171800f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:41.525249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:41.525972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.526246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.526544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] PATH_INFO: `/v3/users/50492499974b4cca87eb680ee3a048b6` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.527655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50492499974b4cca87eb680ee3a048b6)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:41.529534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.542968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343e2af74c434d0f834d29a5fc9092a9'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.543597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343e2af74c434d0f834d29a5fc9092a9'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.553342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343e2af74c434d0f834d29a5fc9092a9'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.565356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343e2af74c434d0f834d29a5fc9092a9'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.571286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 904/1816] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:41 2022] DELETE /identity/v3/users/343e2af74c434d0f834d29a5fc9092a9 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:41.572112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.573410 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.601260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:41.638269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324847b-eb5e-4034-af30-06f293363b8a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:41.643757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:41.644350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.644624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.644887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.645518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:41.645813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:41.647502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.659663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 905/1817] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:41.660615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.662222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.674521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2f930-c01a-490e-9201-a6d2915c431c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:41.679053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95eac5b-9a3c-46f8-a4a0-885f891449a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:41.679659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eac5b-9a3c-46f8-a4a0-885f891449a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.679940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eac5b-9a3c-46f8-a4a0-885f891449a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.680205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95eac5b-9a3c-46f8-a4a0-885f891449a4 admin admin] PATH_INFO: `/v3/projects/a5fbef946d6240bfa7afb17096cd3ce8/users/b043b3de9b9f4fcfaacfed75fae18306/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.695686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eac5b-9a3c-46f8-a4a0-885f891449a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fbef946d6240bfa7afb17096cd3ce8, user_id=b043b3de9b9f4fcfaacfed75fae18306, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:41.698651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95eac5b-9a3c-46f8-a4a0-885f891449a4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.703224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50492499974b4cca87eb680ee3a048b6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.704522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50492499974b4cca87eb680ee3a048b6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.783589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50492499974b4cca87eb680ee3a048b6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.794106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 906/1818] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:41 2022] PUT /identity/v3/projects/a5fbef946d6240bfa7afb17096cd3ce8/users/b043b3de9b9f4fcfaacfed75fae18306/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:41.794106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c8f56-5193-44a5-bc06-3c4a18d4435d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.794106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c8f56-5193-44a5-bc06-3c4a18d4435d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.794106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c8f56-5193-44a5-bc06-3c4a18d4435d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.800720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50492499974b4cca87eb680ee3a048b6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.811120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50492499974b4cca87eb680ee3a048b6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:41.814260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ca1c8f56-5193-44a5-bc06-3c4a18d4435d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:41.815541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ca1c8f56-5193-44a5-bc06-3c4a18d4435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.818920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 913/1819] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:41 2022] DELETE /identity/v3/users/50492499974b4cca87eb680ee3a048b6 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:41.820055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.821960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.883601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 907/1820] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:41.883601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677f4c6-1e8c-431f-9dd0-5980c866ef57 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.883601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677f4c6-1e8c-431f-9dd0-5980c866ef57 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.883601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677f4c6-1e8c-431f-9dd0-5980c866ef57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.902428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6677f4c6-1e8c-431f-9dd0-5980c866ef57 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:41.903635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6677f4c6-1e8c-431f-9dd0-5980c866ef57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.911510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 908/1821] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:41.912698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096483-6c66-4475-8448-69f4696e115a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.912995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096483-6c66-4475-8448-69f4696e115a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.913655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad096483-6c66-4475-8448-69f4696e115a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.917500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b160c7-4259-4c61-acc6-7f120792eec2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:41.923528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e95b95-e516-4e20-8a6f-b641854e3c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:41.924134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e95b95-e516-4e20-8a6f-b641854e3c42 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:41.925389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e95b95-e516-4e20-8a6f-b641854e3c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:41.925389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e95b95-e516-4e20-8a6f-b641854e3c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:41.925389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e95b95-e516-4e20-8a6f-b641854e3c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:41.927238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e95b95-e516-4e20-8a6f-b641854e3c42 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:41.931808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ad096483-6c66-4475-8448-69f4696e115a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:41.933013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ad096483-6c66-4475-8448-69f4696e115a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:41.941420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 909/1822] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:41.942272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad096483-6c66-4475-8448-69f4696e115a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:41.946025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ad096483-6c66-4475-8448-69f4696e115a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:42.003002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad096483-6c66-4475-8448-69f4696e115a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.006599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8408f67-7fc5-4e5a-976b-6f07c61fe9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.007200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8408f67-7fc5-4e5a-976b-6f07c61fe9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.007474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8408f67-7fc5-4e5a-976b-6f07c61fe9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.007741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8408f67-7fc5-4e5a-976b-6f07c61fe9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.008532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 914/1823] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:41 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:42.009371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e95b95-e516-4e20-8a6f-b641854e3c42 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:42.021583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8408f67-7fc5-4e5a-976b-6f07c61fe9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.023287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8408f67-7fc5-4e5a-976b-6f07c61fe9a6 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.034216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 910/1824] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:42.035132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8408f67-7fc5-4e5a-976b-6f07c61fe9a6 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:42.044066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8408f67-7fc5-4e5a-976b-6f07c61fe9a6 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.069447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e95b95-e516-4e20-8a6f-b641854e3c42 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.073939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1f148-17d8-4db8-8afa-dd08cb46505f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.075017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1f148-17d8-4db8-8afa-dd08cb46505f service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.075017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1f148-17d8-4db8-8afa-dd08cb46505f service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.075430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1f148-17d8-4db8-8afa-dd08cb46505f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.087269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:42.105653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c43bb-e114-4179-b948-b1d075e500d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.106705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c43bb-e114-4179-b948-b1d075e500d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.106705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c43bb-e114-4179-b948-b1d075e500d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.107040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c43bb-e114-4179-b948-b1d075e500d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.156788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1f148-17d8-4db8-8afa-dd08cb46505f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.157083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1f148-17d8-4db8-8afa-dd08cb46505f service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.170336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 915/1825] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:42.180219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1f148-17d8-4db8-8afa-dd08cb46505f service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:42.187483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c43bb-e114-4179-b948-b1d075e500d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.189342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c43bb-e114-4179-b948-b1d075e500d3 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.194494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1f148-17d8-4db8-8afa-dd08cb46505f service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.200125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee51a06-6edc-474f-8b54-cde5b58fe540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.202378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 911/1826] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:42.202503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51a06-6edc-474f-8b54-cde5b58fe540 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.202503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51a06-6edc-474f-8b54-cde5b58fe540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.202503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee51a06-6edc-474f-8b54-cde5b58fe540 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.202503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee51a06-6edc-474f-8b54-cde5b58fe540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.206695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee51a06-6edc-474f-8b54-cde5b58fe540 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.322818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 916/1827] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:42 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:42.328701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c43bb-e114-4179-b948-b1d075e500d3 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:42.349828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:42.352904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c43bb-e114-4179-b948-b1d075e500d3 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.358897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553673c-6714-494d-aadb-ee85e5e5efaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.361600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553673c-6714-494d-aadb-ee85e5e5efaf admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.361600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553673c-6714-494d-aadb-ee85e5e5efaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.361600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553673c-6714-494d-aadb-ee85e5e5efaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.361939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553673c-6714-494d-aadb-ee85e5e5efaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.366324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553673c-6714-494d-aadb-ee85e5e5efaf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.386582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 912/1828] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:42.389778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee51a06-6edc-474f-8b54-cde5b58fe540 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:42.403574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee51a06-6edc-474f-8b54-cde5b58fe540 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.419608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.425199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.425494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.425767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.426503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.428332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.441756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 917/1829] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:42.446247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553673c-6714-494d-aadb-ee85e5e5efaf admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:42.458041 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553673c-6714-494d-aadb-ee85e5e5efaf admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.465422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d4d59-4507-41b1-8119-d7be3ad7e2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.466187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4d59-4507-41b1-8119-d7be3ad7e2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.466560 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4d59-4507-41b1-8119-d7be3ad7e2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.466903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d4d59-4507-41b1-8119-d7be3ad7e2e6 admin admin] PATH_INFO: `/v3/projects/5043d10b6bc04c71bd34ce8e876596fe/users/9770d1e9ef7a46c2bba031140097cdc1/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.484286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d4d59-4507-41b1-8119-d7be3ad7e2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5043d10b6bc04c71bd34ce8e876596fe, user_id=9770d1e9ef7a46c2bba031140097cdc1, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.487306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d4d59-4507-41b1-8119-d7be3ad7e2e6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.619984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 913/1830] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:42 2022] PUT /identity/v3/projects/5043d10b6bc04c71bd34ce8e876596fe/users/9770d1e9ef7a46c2bba031140097cdc1/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:42.629275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:42.640913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:42.642554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f010a-a8b3-4e55-a3e7-f8a5000392a3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.643277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f010a-a8b3-4e55-a3e7-f8a5000392a3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.643880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f010a-a8b3-4e55-a3e7-f8a5000392a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.667977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0af63-20ab-46e7-a258-72210e667384 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.706972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-070f010a-a8b3-4e55-a3e7-f8a5000392a3 None None] MFA Rules not processed for user `9770d1e9ef7a46c2bba031140097cdc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:42.710329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-070f010a-a8b3-4e55-a3e7-f8a5000392a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:42.787636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.788450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.788790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.795337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] PATH_INFO: `/v3/projects/6a635c7199ed43b2847cefa75cb1631b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.795337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a635c7199ed43b2847cefa75cb1631b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.795337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.855460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 914/1831] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:42.866714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f010a-a8b3-4e55-a3e7-f8a5000392a3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:42.890591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f010a-a8b3-4e55-a3e7-f8a5000392a3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:42.896712 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:42.897397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:42.897679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:42.897954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:42.898676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:42.918969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:42.963118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a635c7199ed43b2847cefa75cb1631b', 'request_id': 'req-540250a0-2202-488c-a1f6-aa6702c5348c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.013584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 915/1832] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:42 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:43.014969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:43.019092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:43.067767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a635c7199ed43b2847cefa75cb1631b', 'request_id': 'req-540250a0-2202-488c-a1f6-aa6702c5348c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.087932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a635c7199ed43b2847cefa75cb1631b', 'request_id': 'req-540250a0-2202-488c-a1f6-aa6702c5348c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.146814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] The token cache is being invalidate because project 6a635c7199ed43b2847cefa75cb1631b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:43.147368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.148503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.150538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 918/1833] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:42 2022] DELETE /identity/v3/projects/6a635c7199ed43b2847cefa75cb1631b => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:43.152906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:43.155239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:43.168081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b67594-e8b2-4696-ad42-34300efdf624 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:43.195593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:43.198781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:43.199118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:43.202467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] PATH_INFO: `/v3/projects/73901ec50eb6483fbcc980ced4eed624` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:43.205498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73901ec50eb6483fbcc980ced4eed624)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:43.209671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:43.315561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540250a0-2202-488c-a1f6-aa6702c5348c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:43.328619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:43.329787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:43.330234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:43.332118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:43.332914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:43.335202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:43.434578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73901ec50eb6483fbcc980ced4eed624', 'request_id': 'req-cd5c4705-aff6-41cf-880f-7e33b0b2a486'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.481933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 919/1834] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:43 2022] POST /identity/v3/users => generated 297 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:43.483185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:43.486161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:43.498697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c9191-6a6e-43f9-93e7-a1f8c41fadc4 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:43.503988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73901ec50eb6483fbcc980ced4eed624', 'request_id': 'req-cd5c4705-aff6-41cf-880f-7e33b0b2a486'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.528071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73901ec50eb6483fbcc980ced4eed624', 'request_id': 'req-cd5c4705-aff6-41cf-880f-7e33b0b2a486'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.544801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] The token cache is being invalidate because project 73901ec50eb6483fbcc980ced4eed624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:43.545936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.546161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.549951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 916/1835] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:43 2022] DELETE /identity/v3/projects/73901ec50eb6483fbcc980ced4eed624 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:43.551522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:43.553389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:43.611488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:43.613077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:43.622250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:43.622690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] PATH_INFO: `/v3/users/6f7725beb38a4bac952870541a6fe5a6` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:43.627171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7725beb38a4bac952870541a6fe5a6)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:43.629145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:43.643482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c4705-aff6-41cf-880f-7e33b0b2a486 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:43.649702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:43.650319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:43.651487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:43.651765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:43.652493 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:43.654202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:43.668878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 917/1836] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:43.668878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:43.670416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:43.680565 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:43.686312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc9aa3-ebf1-4b09-9f38-72fd6bc26dbf admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:43.691919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:43.692540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:43.692815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:43.693081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] PATH_INFO: `/v3/users/27d2f1ae66a446818c0397504bda0056` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:43.694196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d2f1ae66a446818c0397504bda0056)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:43.696186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:43.945204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7725beb38a4bac952870541a6fe5a6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.945920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7725beb38a4bac952870541a6fe5a6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:43.960287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d2f1ae66a446818c0397504bda0056'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:44.004506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:44.037672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7725beb38a4bac952870541a6fe5a6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:44.052321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7725beb38a4bac952870541a6fe5a6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:44.059201 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7725beb38a4bac952870541a6fe5a6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:44.065365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 920/1837] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:43 2022] DELETE /identity/v3/users/6f7725beb38a4bac952870541a6fe5a6 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:44.066145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:44.067492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:44.112673 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d2f1ae66a446818c0397504bda0056'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:44.113313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d2f1ae66a446818c0397504bda0056'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:44.121343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d2f1ae66a446818c0397504bda0056'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:44.134715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-dac32807-4110-4a51-94e1-89c37467d509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d2f1ae66a446818c0397504bda0056'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:44.139042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f672e-d40d-416a-b3f6-6f2e14afcd2f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:44.145186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 918/1838] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:43 2022] DELETE /identity/v3/users/27d2f1ae66a446818c0397504bda0056 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:44.234032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:44.235010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:44.235010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:44.235303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] PATH_INFO: `/v3/projects/2762430dc7824dae95bf74bf40eaf9d6/users/c0e63327bdf4411f8c9fe4b93ef3a38b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:44.261212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2762430dc7824dae95bf74bf40eaf9d6, user_id=c0e63327bdf4411f8c9fe4b93ef3a38b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:44.264203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:44.291950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511f9a-1c8f-42e3-98d0-4ff79eab8c07 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:44.292149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511f9a-1c8f-42e3-98d0-4ff79eab8c07 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:44.292398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc511f9a-1c8f-42e3-98d0-4ff79eab8c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:44.332393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fc511f9a-1c8f-42e3-98d0-4ff79eab8c07 None None] MFA Rules not processed for user `b043b3de9b9f4fcfaacfed75fae18306`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:44.333839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fc511f9a-1c8f-42e3-98d0-4ff79eab8c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:44.389307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 921/1839] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:44 2022] PUT /identity/v3/projects/2762430dc7824dae95bf74bf40eaf9d6/users/c0e63327bdf4411f8c9fe4b93ef3a38b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:44.396591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:44.398632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:44.447346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 919/1840] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:44.451855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc511f9a-1c8f-42e3-98d0-4ff79eab8c07 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:44.453258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fc511f9a-1c8f-42e3-98d0-4ff79eab8c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:44.493695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc511f9a-1c8f-42e3-98d0-4ff79eab8c07 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:44.540494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20b08f-e323-4ef0-b1a5-58dc113a24f9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:44.545867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699b3c1-0f37-4f0b-bca1-d4c7a8368095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:44.546796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699b3c1-0f37-4f0b-bca1-d4c7a8368095 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:44.546796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699b3c1-0f37-4f0b-bca1-d4c7a8368095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:44.547787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699b3c1-0f37-4f0b-bca1-d4c7a8368095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:44.555041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699b3c1-0f37-4f0b-bca1-d4c7a8368095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:44.557617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699b3c1-0f37-4f0b-bca1-d4c7a8368095 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:44.572997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 922/1841] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:44.576981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699b3c1-0f37-4f0b-bca1-d4c7a8368095 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:44.594225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699b3c1-0f37-4f0b-bca1-d4c7a8368095 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:44.605071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2be9a-18bd-482a-917b-bf01f7e57bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:44.606060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2be9a-18bd-482a-917b-bf01f7e57bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:44.606324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2be9a-18bd-482a-917b-bf01f7e57bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:44.606922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2be9a-18bd-482a-917b-bf01f7e57bd3 admin admin] PATH_INFO: `/v3/projects/2762430dc7824dae95bf74bf40eaf9d6/users/c0e63327bdf4411f8c9fe4b93ef3a38b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:44.609165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2be9a-18bd-482a-917b-bf01f7e57bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2762430dc7824dae95bf74bf40eaf9d6, user_id=c0e63327bdf4411f8c9fe4b93ef3a38b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:44.614141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2be9a-18bd-482a-917b-bf01f7e57bd3 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:44.626207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b322d2-eb79-48fb-8982-5e85f377d13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:44.626670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b322d2-eb79-48fb-8982-5e85f377d13d admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:44.626866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b322d2-eb79-48fb-8982-5e85f377d13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:44.629961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b322d2-eb79-48fb-8982-5e85f377d13d admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:44.629961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b322d2-eb79-48fb-8982-5e85f377d13d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:44.629961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b322d2-eb79-48fb-8982-5e85f377d13d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:44.630605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 923/1842] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:44 2022] PUT /identity/v3/projects/2762430dc7824dae95bf74bf40eaf9d6/users/c0e63327bdf4411f8c9fe4b93ef3a38b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:44.646734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0159662-a2f7-4986-a916-1e5180c445cf None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:44.647471 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0159662-a2f7-4986-a916-1e5180c445cf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:44.647968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0159662-a2f7-4986-a916-1e5180c445cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:44.715285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d0159662-a2f7-4986-a916-1e5180c445cf None None] MFA Rules not processed for user `c0e63327bdf4411f8c9fe4b93ef3a38b`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:44.719900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d0159662-a2f7-4986-a916-1e5180c445cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:44.767545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 920/1843] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:44 2022] POST /identity/v3/projects => generated 373 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:44.772138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b322d2-eb79-48fb-8982-5e85f377d13d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:44.842470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 924/1844] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:44.854900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0159662-a2f7-4986-a916-1e5180c445cf None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:44.857602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d0159662-a2f7-4986-a916-1e5180c445cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:44.866786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b322d2-eb79-48fb-8982-5e85f377d13d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:44.876524 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba640b-a290-4423-8738-a6cca5166084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:44.877432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba640b-a290-4423-8738-a6cca5166084 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:44.877709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba640b-a290-4423-8738-a6cca5166084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:44.878293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba640b-a290-4423-8738-a6cca5166084 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:44.878987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba640b-a290-4423-8738-a6cca5166084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:44.880992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba640b-a290-4423-8738-a6cca5166084 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:44.898264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:44.975584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0159662-a2f7-4986-a916-1e5180c445cf None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:45.026913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 921/1845] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:44 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:45.027875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba640b-a290-4423-8738-a6cca5166084 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:45.051103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba640b-a290-4423-8738-a6cca5166084 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:45.062317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769bfa2-8e57-4ecc-b800-5915d2734c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:45.063747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769bfa2-8e57-4ecc-b800-5915d2734c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:45.064247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769bfa2-8e57-4ecc-b800-5915d2734c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:45.067022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769bfa2-8e57-4ecc-b800-5915d2734c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:45.067720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769bfa2-8e57-4ecc-b800-5915d2734c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:45.071406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769bfa2-8e57-4ecc-b800-5915d2734c32 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:45.093313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:45.094539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:45.094893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:45.095363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:45.102865 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 922/1846] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:45.108882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769bfa2-8e57-4ecc-b800-5915d2734c32 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:45.134287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769bfa2-8e57-4ecc-b800-5915d2734c32 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:45.155398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:45.155398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:45.155398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:45.155398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] PATH_INFO: `/v3/projects/cfa2980ed0544c389f1fe3a463546033/users/58718ac434f64aceba4262577d157d0d/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:45.198307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa2980ed0544c389f1fe3a463546033, user_id=58718ac434f64aceba4262577d157d0d, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:45.205613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:45.238507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:45.240072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:45.252605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 925/1847] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:21:45.253439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:45.258466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:45.271890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56539cb1-adda-4bb3-a675-956bcd6e2fbf service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:45.278216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:45.278991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:45.279418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:45.279741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] PATH_INFO: `/v3/projects/c76ada4473804b96abce6c05b23bcdad` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:45.280972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76ada4473804b96abce6c05b23bcdad)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:45.283904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:45.289203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:45.375333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 923/1848] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:45 2022] PUT /identity/v3/projects/cfa2980ed0544c389f1fe3a463546033/users/58718ac434f64aceba4262577d157d0d/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:45.376322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:45.379784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:45.442741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76ada4473804b96abce6c05b23bcdad', 'request_id': 'req-42392cad-0aee-44a2-96d2-7e171d6ed142'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.510393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346e2dc-ec1e-4ecc-a69d-60885768945b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:45.526382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76ada4473804b96abce6c05b23bcdad', 'request_id': 'req-42392cad-0aee-44a2-96d2-7e171d6ed142'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.537356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76ada4473804b96abce6c05b23bcdad', 'request_id': 'req-42392cad-0aee-44a2-96d2-7e171d6ed142'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.562196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] The token cache is being invalidate because project c76ada4473804b96abce6c05b23bcdad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:45.562554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.564272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.569412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 926/1849] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:45 2022] DELETE /identity/v3/projects/c76ada4473804b96abce6c05b23bcdad => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:45.570812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:45.572220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:45.696209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:45.698815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:45.699100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:45.699466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] PATH_INFO: `/v3/projects/266ff0f2a34049d288124ce9d545dd7a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:45.703985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=266ff0f2a34049d288124ce9d545dd7a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:45.709179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:45.754940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42392cad-0aee-44a2-96d2-7e171d6ed142 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:45.760427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:45.761258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:45.761681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:45.762121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:45.762953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:45.764857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:45.775496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 927/1850] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:45 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:45.776322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:45.778871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:45.850434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266ff0f2a34049d288124ce9d545dd7a', 'request_id': 'req-d480685f-7945-426e-8419-a1c41c660c1a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.852263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42546c83-7ab4-447b-9189-2af03ad4acdd admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:45.906163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:45.907494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:45.907764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:45.908032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:45.909932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:45.931961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266ff0f2a34049d288124ce9d545dd7a', 'request_id': 'req-d480685f-7945-426e-8419-a1c41c660c1a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.938169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266ff0f2a34049d288124ce9d545dd7a', 'request_id': 'req-d480685f-7945-426e-8419-a1c41c660c1a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.960587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] The token cache is being invalidate because project 266ff0f2a34049d288124ce9d545dd7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:45.960911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.961766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d480685f-7945-426e-8419-a1c41c660c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:45.965629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 924/1851] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:45 2022] DELETE /identity/v3/projects/266ff0f2a34049d288124ce9d545dd7a => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:45.965629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d34aea-e9ca-4c17-92ad-ca430e437062 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:45.965629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d34aea-e9ca-4c17-92ad-ca430e437062 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:45.965629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d34aea-e9ca-4c17-92ad-ca430e437062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:45.988516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-11d34aea-e9ca-4c17-92ad-ca430e437062 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:45.989753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-11d34aea-e9ca-4c17-92ad-ca430e437062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:45.997144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:45.998836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.008894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 928/1852] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:46.009861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.077855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 925/1853] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:46.079815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25845-a8be-4160-a52e-9b2eff931c73 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.080215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25845-a8be-4160-a52e-9b2eff931c73 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.080607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d25845-a8be-4160-a52e-9b2eff931c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.100506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbffc8-afb7-4f06-8198-b0eaf0a56587 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.107838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506566f-923f-4f0d-a4cc-6da60aff64dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.108703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506566f-923f-4f0d-a4cc-6da60aff64dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.108819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-72d25845-a8be-4160-a52e-9b2eff931c73 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:46.114404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506566f-923f-4f0d-a4cc-6da60aff64dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.114404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506566f-923f-4f0d-a4cc-6da60aff64dc admin admin] PATH_INFO: `/v3/projects/cfa2980ed0544c389f1fe3a463546033/users/58718ac434f64aceba4262577d157d0d/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.114796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-72d25845-a8be-4160-a52e-9b2eff931c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:46.114886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506566f-923f-4f0d-a4cc-6da60aff64dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa2980ed0544c389f1fe3a463546033, user_id=58718ac434f64aceba4262577d157d0d, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.117075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506566f-923f-4f0d-a4cc-6da60aff64dc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.133922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 926/1855] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:46.143133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f3b0-4f01-424d-9ba2-108a0ad8c4b2 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.143133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f3b0-4f01-424d-9ba2-108a0ad8c4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.143133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f3b0-4f01-424d-9ba2-108a0ad8c4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.158445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 929/1855] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:46 2022] PUT /identity/v3/projects/cfa2980ed0544c389f1fe3a463546033/users/58718ac434f64aceba4262577d157d0d/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:46.163098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72db23c-415d-47e3-b730-4a5ad934d13e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.163996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72db23c-415d-47e3-b730-4a5ad934d13e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.163996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72db23c-415d-47e3-b730-4a5ad934d13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.176154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4b77f3b0-4f01-424d-9ba2-108a0ad8c4b2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:46.177933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4b77f3b0-4f01-424d-9ba2-108a0ad8c4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:46.208827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d72db23c-415d-47e3-b730-4a5ad934d13e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:46.210052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d72db23c-415d-47e3-b730-4a5ad934d13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:46.344632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 930/1856] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:46.346506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72db23c-415d-47e3-b730-4a5ad934d13e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.357299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 927/1857] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:46.361393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77f3b0-4f01-424d-9ba2-108a0ad8c4b2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.367013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72db23c-415d-47e3-b730-4a5ad934d13e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.384020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77f3b0-4f01-424d-9ba2-108a0ad8c4b2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.389899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.393136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.393427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.393700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.396497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:46.396834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.398911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.399770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.400193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.400732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.401739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:46.402199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.402663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.404217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.409179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:46.425239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 931/1858] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:46.429858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.443740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 928/1859] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:21:46.453031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.461516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fb0e8-490d-4b48-9bc7-657ada849a32 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.465585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771fe12-5f1f-4a4e-bbb6-aa222d35a64a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.473113 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e800-75e8-48bd-9773-d97cb49c8c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.473919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e800-75e8-48bd-9773-d97cb49c8c27 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.474429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e800-75e8-48bd-9773-d97cb49c8c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.475036 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01126467-f82b-4dfd-9bb6-c829f0a11cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.475247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1e800-75e8-48bd-9773-d97cb49c8c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.475661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01126467-f82b-4dfd-9bb6-c829f0a11cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.475939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01126467-f82b-4dfd-9bb6-c829f0a11cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.476207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01126467-f82b-4dfd-9bb6-c829f0a11cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.476596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1e800-75e8-48bd-9773-d97cb49c8c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.476893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01126467-f82b-4dfd-9bb6-c829f0a11cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.479648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1e800-75e8-48bd-9773-d97cb49c8c27 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.480555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01126467-f82b-4dfd-9bb6-c829f0a11cbf admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.588098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 929/1860] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:46 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:46.588810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 932/1861] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:46 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:46.596868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01126467-f82b-4dfd-9bb6-c829f0a11cbf admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.606134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e800-75e8-48bd-9773-d97cb49c8c27 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.728370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1e800-75e8-48bd-9773-d97cb49c8c27 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.729281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01126467-f82b-4dfd-9bb6-c829f0a11cbf admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.737390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fc674-3b66-4198-8960-53593d4e465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.737390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc674-3b66-4198-8960-53593d4e465d admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.737390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc674-3b66-4198-8960-53593d4e465d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.737702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5f50e-f0d6-4ebe-aa24-022a199e3fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.737803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fc674-3b66-4198-8960-53593d4e465d admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.737803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc674-3b66-4198-8960-53593d4e465d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.738332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5f50e-f0d6-4ebe-aa24-022a199e3fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.738733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5f50e-f0d6-4ebe-aa24-022a199e3fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.739109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5f50e-f0d6-4ebe-aa24-022a199e3fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.739579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fc674-3b66-4198-8960-53593d4e465d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.739947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5f50e-f0d6-4ebe-aa24-022a199e3fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.742020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5f50e-f0d6-4ebe-aa24-022a199e3fd6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.864683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 933/1862] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:46 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:46.868664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5f50e-f0d6-4ebe-aa24-022a199e3fd6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.880523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 930/1863] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:46 2022] POST /identity/v3/users => generated 292 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:46.887039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5f50e-f0d6-4ebe-aa24-022a199e3fd6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.892333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fc674-3b66-4198-8960-53593d4e465d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:46.896546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5564e7-1c05-43cb-bae7-404be99b9241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.897615 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5564e7-1c05-43cb-bae7-404be99b9241 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.900250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5564e7-1c05-43cb-bae7-404be99b9241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.900722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5564e7-1c05-43cb-bae7-404be99b9241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.902825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5564e7-1c05-43cb-bae7-404be99b9241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.908977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5564e7-1c05-43cb-bae7-404be99b9241 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.919998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fc674-3b66-4198-8960-53593d4e465d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:46.951783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f67b0-30e9-4775-a95e-5d9fb8424458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:46.952421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f67b0-30e9-4775-a95e-5d9fb8424458 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:46.953068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f67b0-30e9-4775-a95e-5d9fb8424458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:46.953068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f67b0-30e9-4775-a95e-5d9fb8424458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:46.954968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f67b0-30e9-4775-a95e-5d9fb8424458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:46.971182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f67b0-30e9-4775-a95e-5d9fb8424458 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:46.983421 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 934/1864] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:21:46.986625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5564e7-1c05-43cb-bae7-404be99b9241 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.027333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5564e7-1c05-43cb-bae7-404be99b9241 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.028038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 931/1865] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:47.032279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f67b0-30e9-4775-a95e-5d9fb8424458 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.034718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b1302-61b4-4d3c-b801-b798000b87f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.035337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b1302-61b4-4d3c-b801-b798000b87f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.035614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b1302-61b4-4d3c-b801-b798000b87f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.036603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b1302-61b4-4d3c-b801-b798000b87f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.037451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b1302-61b4-4d3c-b801-b798000b87f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:47.039181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b1302-61b4-4d3c-b801-b798000b87f1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:47.068176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f67b0-30e9-4775-a95e-5d9fb8424458 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.079642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 935/1866] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:21:47.083591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b1302-61b4-4d3c-b801-b798000b87f1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.092956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cbd03-3d6a-42af-b2bc-4fb76e22da9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.093791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cbd03-3d6a-42af-b2bc-4fb76e22da9f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.094753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cbd03-3d6a-42af-b2bc-4fb76e22da9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.095112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cbd03-3d6a-42af-b2bc-4fb76e22da9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.096035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cbd03-3d6a-42af-b2bc-4fb76e22da9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:47.097740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cbd03-3d6a-42af-b2bc-4fb76e22da9f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:47.108230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b1302-61b4-4d3c-b801-b798000b87f1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.116953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c834c-4acf-4835-9e70-122493a69d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.117946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c834c-4acf-4835-9e70-122493a69d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.118400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c834c-4acf-4835-9e70-122493a69d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.119347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c834c-4acf-4835-9e70-122493a69d5b admin admin] PATH_INFO: `/v3/projects/fbeb6967bdca4ee4a5fc0c7d6b9416ce/users/474b0b5dfa7a4a97b8fb1f8d6e014974/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.123115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 932/1867] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:47.127010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cbd03-3d6a-42af-b2bc-4fb76e22da9f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.139212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cbd03-3d6a-42af-b2bc-4fb76e22da9f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.146768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53270539-f441-4b8e-a64c-e43bf63944eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.147610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53270539-f441-4b8e-a64c-e43bf63944eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.147954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53270539-f441-4b8e-a64c-e43bf63944eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.148311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53270539-f441-4b8e-a64c-e43bf63944eb admin admin] PATH_INFO: `/v3/projects/c31116d92493460793d41ba2a6a166e0/users/72467431e40c491cad192b06be494175/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.159088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c834c-4acf-4835-9e70-122493a69d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbeb6967bdca4ee4a5fc0c7d6b9416ce, user_id=474b0b5dfa7a4a97b8fb1f8d6e014974, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:47.162914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c834c-4acf-4835-9e70-122493a69d5b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:47.168643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53270539-f441-4b8e-a64c-e43bf63944eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31116d92493460793d41ba2a6a166e0, user_id=72467431e40c491cad192b06be494175, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:47.172209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53270539-f441-4b8e-a64c-e43bf63944eb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:47.258334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 936/1868] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:47 2022] PUT /identity/v3/projects/fbeb6967bdca4ee4a5fc0c7d6b9416ce/users/474b0b5dfa7a4a97b8fb1f8d6e014974/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:47.271132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0375-689e-42b2-8c71-48a3bf6f43e6 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.271409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0375-689e-42b2-8c71-48a3bf6f43e6 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.271711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed0375-689e-42b2-8c71-48a3bf6f43e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.280928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 933/1869] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:47 2022] PUT /identity/v3/projects/c31116d92493460793d41ba2a6a166e0/users/72467431e40c491cad192b06be494175/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:47.286607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c4efb-f6fd-459b-9c0e-aa3e9ccc4633 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.286892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c4efb-f6fd-459b-9c0e-aa3e9ccc4633 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.287162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c4efb-f6fd-459b-9c0e-aa3e9ccc4633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.314790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e6ed0375-689e-42b2-8c71-48a3bf6f43e6 None None] MFA Rules not processed for user `474b0b5dfa7a4a97b8fb1f8d6e014974`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:47.316176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e6ed0375-689e-42b2-8c71-48a3bf6f43e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:47.333901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3f1c4efb-f6fd-459b-9c0e-aa3e9ccc4633 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:47.335339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3f1c4efb-f6fd-459b-9c0e-aa3e9ccc4633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:47.393671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 937/1870] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:47.398170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed0375-689e-42b2-8c71-48a3bf6f43e6 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.456527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 934/1871] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:47.467317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c4efb-f6fd-459b-9c0e-aa3e9ccc4633 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.469327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3f1c4efb-f6fd-459b-9c0e-aa3e9ccc4633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:47.491168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed0375-689e-42b2-8c71-48a3bf6f43e6 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.504747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbeebd-d306-4ee3-8d5a-87fedf47c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.505370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbeebd-d306-4ee3-8d5a-87fedf47c245 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.505648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbeebd-d306-4ee3-8d5a-87fedf47c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.506489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dbeebd-d306-4ee3-8d5a-87fedf47c245 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.507440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbeebd-d306-4ee3-8d5a-87fedf47c245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:47.509085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dbeebd-d306-4ee3-8d5a-87fedf47c245 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:47.567889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c4efb-f6fd-459b-9c0e-aa3e9ccc4633 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.571898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bec4c-ed8a-414d-9fc3-71b7dc66c12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.572558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bec4c-ed8a-414d-9fc3-71b7dc66c12d service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.579170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bec4c-ed8a-414d-9fc3-71b7dc66c12d service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.579170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219bec4c-ed8a-414d-9fc3-71b7dc66c12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.583982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bec4c-ed8a-414d-9fc3-71b7dc66c12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:47.585457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219bec4c-ed8a-414d-9fc3-71b7dc66c12d service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:47.597212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 935/1872] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:47.682303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 938/1873] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:47 2022] POST /identity/v3/projects => generated 385 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:47.686787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bec4c-ed8a-414d-9fc3-71b7dc66c12d service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.707904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219bec4c-ed8a-414d-9fc3-71b7dc66c12d service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.789298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410427e-033c-466a-b877-68545a4fcf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.789948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410427e-033c-466a-b877-68545a4fcf00 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.790195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410427e-033c-466a-b877-68545a4fcf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.790621 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410427e-033c-466a-b877-68545a4fcf00 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.791172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410427e-033c-466a-b877-68545a4fcf00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:47.793170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410427e-033c-466a-b877-68545a4fcf00 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:47.850499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:47.927070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 936/1874] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:47 2022] POST /identity/v3/users => generated 299 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:47.930893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbeebd-d306-4ee3-8d5a-87fedf47c245 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.941893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dbeebd-d306-4ee3-8d5a-87fedf47c245 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.946399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8cc11-1d89-404c-877b-2bad2c3df9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.947073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8cc11-1d89-404c-877b-2bad2c3df9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.947349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8cc11-1d89-404c-877b-2bad2c3df9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.947633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8cc11-1d89-404c-877b-2bad2c3df9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.948273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8cc11-1d89-404c-877b-2bad2c3df9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:47.949919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8cc11-1d89-404c-877b-2bad2c3df9a6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:47.963962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 939/1875] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:21:47.965365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410427e-033c-466a-b877-68545a4fcf00 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:47.975914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410427e-033c-466a-b877-68545a4fcf00 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:47.982511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05938a-d198-4c1e-a65f-ea91a33dbcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:47.982675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05938a-d198-4c1e-a65f-ea91a33dbcc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:47.982934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05938a-d198-4c1e-a65f-ea91a33dbcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:47.983287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05938a-d198-4c1e-a65f-ea91a33dbcc7 admin admin] PATH_INFO: `/v3/projects/a5bb6eff9887465999946e8e5fb0712b/users/2146ac38f15f4102b397c9ddf9592d72/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:47.999526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05938a-d198-4c1e-a65f-ea91a33dbcc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bb6eff9887465999946e8e5fb0712b, user_id=2146ac38f15f4102b397c9ddf9592d72, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:48.003047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05938a-d198-4c1e-a65f-ea91a33dbcc7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:48.095047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 937/1876] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:47 2022] PUT /identity/v3/projects/a5bb6eff9887465999946e8e5fb0712b/users/2146ac38f15f4102b397c9ddf9592d72/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:48.097354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8cc11-1d89-404c-877b-2bad2c3df9a6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:48.123668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8cc11-1d89-404c-877b-2bad2c3df9a6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:48.219785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab2b41-91b6-4d9e-8806-84b6e14dd143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:48.220714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2b41-91b6-4d9e-8806-84b6e14dd143 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:48.227315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2b41-91b6-4d9e-8806-84b6e14dd143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:48.227315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2b41-91b6-4d9e-8806-84b6e14dd143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:48.227315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab2b41-91b6-4d9e-8806-84b6e14dd143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:48.231292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab2b41-91b6-4d9e-8806-84b6e14dd143 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:48.252400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 940/1877] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:48.253704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05938a-d198-4c1e-a65f-ea91a33dbcc7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:48.268337 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05938a-d198-4c1e-a65f-ea91a33dbcc7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:48.291584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:48.292373 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:48.292652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:48.292923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] PATH_INFO: `/v3/projects/a5bb6eff9887465999946e8e5fb0712b/users/2146ac38f15f4102b397c9ddf9592d72/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:48.294936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bb6eff9887465999946e8e5fb0712b, user_id=2146ac38f15f4102b397c9ddf9592d72, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:48.298828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:48.318399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 938/1878] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:48 2022] PUT /identity/v3/projects/a5bb6eff9887465999946e8e5fb0712b/users/2146ac38f15f4102b397c9ddf9592d72/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:48.326265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ffc8a-a2d2-48d3-8e43-07ae7a9364c5 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:48.326265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ffc8a-a2d2-48d3-8e43-07ae7a9364c5 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:48.326265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ffc8a-a2d2-48d3-8e43-07ae7a9364c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:48.368191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c67ffc8a-a2d2-48d3-8e43-07ae7a9364c5 None None] MFA Rules not processed for user `2146ac38f15f4102b397c9ddf9592d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:48.369896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c67ffc8a-a2d2-48d3-8e43-07ae7a9364c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:48.492206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 941/1879] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:48.505467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:48.509360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:48.525873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e43fa-db1b-413a-8523-abe07a853f7b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:48.526160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e43fa-db1b-413a-8523-abe07a853f7b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:48.526460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e43fa-db1b-413a-8523-abe07a853f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:48.577364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-896e43fa-db1b-413a-8523-abe07a853f7b None None] MFA Rules not processed for user `58718ac434f64aceba4262577d157d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:48.600668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f7078-0701-4207-b9bc-d88c0b8b3df6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:48.606851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3a728-cd2b-41ea-afc6-c5ae3b91ef1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:48.608282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3a728-cd2b-41ea-afc6-c5ae3b91ef1a service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:48.608564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3a728-cd2b-41ea-afc6-c5ae3b91ef1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:48.609131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3a728-cd2b-41ea-afc6-c5ae3b91ef1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:48.613579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-896e43fa-db1b-413a-8523-abe07a853f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:48.626810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3a728-cd2b-41ea-afc6-c5ae3b91ef1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:48.628446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3a728-cd2b-41ea-afc6-c5ae3b91ef1a service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:48.644248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 939/1880] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:48.700382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3a728-cd2b-41ea-afc6-c5ae3b91ef1a service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:48.834574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3a728-cd2b-41ea-afc6-c5ae3b91ef1a service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:48.841123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:48.841585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 942/1881] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:48.842507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:48.842949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:48.843903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] PATH_INFO: `/v3/users/474b0b5dfa7a4a97b8fb1f8d6e014974` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:48.845956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=474b0b5dfa7a4a97b8fb1f8d6e014974)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:48.848520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:48.853516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e43fa-db1b-413a-8523-abe07a853f7b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:48.857114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-896e43fa-db1b-413a-8523-abe07a853f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:48.884195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e43fa-db1b-413a-8523-abe07a853f7b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:48.900259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b4a76-ad68-4cbd-bc26-4dff3f1148c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:48.901064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b4a76-ad68-4cbd-bc26-4dff3f1148c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:48.901357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b4a76-ad68-4cbd-bc26-4dff3f1148c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:48.901631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b4a76-ad68-4cbd-bc26-4dff3f1148c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:48.906311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c40b4a76-ad68-4cbd-bc26-4dff3f1148c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:49.043248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b4a76-ad68-4cbd-bc26-4dff3f1148c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:49.045037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b4a76-ad68-4cbd-bc26-4dff3f1148c6 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:49.086380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474b0b5dfa7a4a97b8fb1f8d6e014974'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:49.089978 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 943/1882] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:21:49.206134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391978-93ed-4802-b733-a6c82f726527 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:49.207497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391978-93ed-4802-b733-a6c82f726527 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:49.207778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391978-93ed-4802-b733-a6c82f726527 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:49.211648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 944/1883] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:21:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:21:49.217778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474b0b5dfa7a4a97b8fb1f8d6e014974'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:49.218443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474b0b5dfa7a4a97b8fb1f8d6e014974'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:49.224879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f52b89-6281-4c6b-a32b-a228cf020565 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:49.225170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f52b89-6281-4c6b-a32b-a228cf020565 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:49.227129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474b0b5dfa7a4a97b8fb1f8d6e014974'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:49.231318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f52b89-6281-4c6b-a32b-a228cf020565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:49.241358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30b1394e-2416-4103-af38-5a293ad47d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474b0b5dfa7a4a97b8fb1f8d6e014974'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:49.249649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 940/1884] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:48 2022] DELETE /identity/v3/users/474b0b5dfa7a4a97b8fb1f8d6e014974 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:49.256367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e553-ee85-4138-8c3b-3c5b298ee951 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:49.256803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e553-ee85-4138-8c3b-3c5b298ee951 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:49.257179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958e553-ee85-4138-8c3b-3c5b298ee951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:49.277521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-62f52b89-6281-4c6b-a32b-a228cf020565 None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:49.278714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-62f52b89-6281-4c6b-a32b-a228cf020565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:49.280149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9958e553-ee85-4138-8c3b-3c5b298ee951 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:49.281961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9958e553-ee85-4138-8c3b-3c5b298ee951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:49.347823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 941/1885] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:49.357285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 945/1886] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:21:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:49.362550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958e553-ee85-4138-8c3b-3c5b298ee951 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:49.363964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9958e553-ee85-4138-8c3b-3c5b298ee951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:49.410521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f52b89-6281-4c6b-a32b-a228cf020565 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:49.410521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-62f52b89-6281-4c6b-a32b-a228cf020565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:49.431775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f52b89-6281-4c6b-a32b-a228cf020565 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:49.441267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5c518-3171-409e-9f02-093a65db54fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:49.441890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c518-3171-409e-9f02-093a65db54fe service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:49.442164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c518-3171-409e-9f02-093a65db54fe service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:49.442462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5c518-3171-409e-9f02-093a65db54fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:49.465022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958e553-ee85-4138-8c3b-3c5b298ee951 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:49.469913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:49.470984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:49.470984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:49.470984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:49.477935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5c518-3171-409e-9f02-093a65db54fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:49.479837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5c518-3171-409e-9f02-093a65db54fe service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:49.483139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:49.484799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:49.492191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 946/1887] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:49.495675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 942/1888] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:49.716593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82f64b-50b8-4885-b883-ecfa9576ab8a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:49.716593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82f64b-50b8-4885-b883-ecfa9576ab8a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:49.716593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82f64b-50b8-4885-b883-ecfa9576ab8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:49.750298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2c82f64b-50b8-4885-b883-ecfa9576ab8a None None] MFA Rules not processed for user `72467431e40c491cad192b06be494175`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:49.752046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2c82f64b-50b8-4885-b883-ecfa9576ab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:49.810917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:49.815900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:49.867406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620bc57-aebc-4fe8-9bfa-ca5f44278f53 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:49.878020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:49.878827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:49.879108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:49.879382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] PATH_INFO: `/v3/users/9770d1e9ef7a46c2bba031140097cdc1` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:49.881033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9770d1e9ef7a46c2bba031140097cdc1)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:49.883834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:49.921757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 947/1889] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:49.945180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82f64b-50b8-4885-b883-ecfa9576ab8a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:49.948889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2c82f64b-50b8-4885-b883-ecfa9576ab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:50.049808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82f64b-50b8-4885-b883-ecfa9576ab8a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:50.068379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:50.068379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:50.068379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:50.068379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:50.070795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:50.072321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:50.082187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 948/1890] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:50.130506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9770d1e9ef7a46c2bba031140097cdc1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.188040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:50.190044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:50.213123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfd6dc4-6744-4793-980c-cdc16f591e06 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:50.221190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:50.222280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:50.222280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:50.223040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] PATH_INFO: `/v3/projects/fbeb6967bdca4ee4a5fc0c7d6b9416ce` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:50.238119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbeb6967bdca4ee4a5fc0c7d6b9416ce)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:50.240940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:50.269820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9770d1e9ef7a46c2bba031140097cdc1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.275054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9770d1e9ef7a46c2bba031140097cdc1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.281105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9770d1e9ef7a46c2bba031140097cdc1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.305825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8ad91371-e087-43fe-a0d1-cf789998aff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9770d1e9ef7a46c2bba031140097cdc1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.313470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 943/1891] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:49 2022] DELETE /identity/v3/users/9770d1e9ef7a46c2bba031140097cdc1 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:50.317677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e137de-f574-491f-a811-2bbb392da47b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:50.318071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e137de-f574-491f-a811-2bbb392da47b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:50.318569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e137de-f574-491f-a811-2bbb392da47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:50.343804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-31e137de-f574-491f-a811-2bbb392da47b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:50.345463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-31e137de-f574-491f-a811-2bbb392da47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:50.416493 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 944/1892] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:50.425047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e137de-f574-491f-a811-2bbb392da47b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:50.426977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-31e137de-f574-491f-a811-2bbb392da47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:50.449777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbeb6967bdca4ee4a5fc0c7d6b9416ce', 'request_id': 'req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.500283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e137de-f574-491f-a811-2bbb392da47b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:50.541708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbeb6967bdca4ee4a5fc0c7d6b9416ce', 'request_id': 'req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.555061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbeb6967bdca4ee4a5fc0c7d6b9416ce', 'request_id': 'req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.558909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:50.559089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:50.559316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:50.559588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:50.578994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] The token cache is being invalidate because project fbeb6967bdca4ee4a5fc0c7d6b9416ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:50.580262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.584861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.586967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 949/1893] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:50 2022] DELETE /identity/v3/projects/fbeb6967bdca4ee4a5fc0c7d6b9416ce => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:50.591214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:50.595355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:50.671539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:50.672822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:50.685726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 945/1894] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:50.720388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c8518-25e8-4ef7-a2c9-46b03cadaa6e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:50.732318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:50.732318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:50.732318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:50.732318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] PATH_INFO: `/v3/users/2146ac38f15f4102b397c9ddf9592d72` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:50.732765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2146ac38f15f4102b397c9ddf9592d72)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:50.732765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:50.750655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:50.969532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2146ac38f15f4102b397c9ddf9592d72'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:50.970325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2146ac38f15f4102b397c9ddf9592d72'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.076881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2146ac38f15f4102b397c9ddf9592d72'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.078169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:51.079968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:51.101989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2146ac38f15f4102b397c9ddf9592d72'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.118296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2146ac38f15f4102b397c9ddf9592d72'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.137362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 950/1895] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:50 2022] DELETE /identity/v3/users/2146ac38f15f4102b397c9ddf9592d72 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:51.200331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1aa20e-a38b-4db1-b579-ac5710e6e977 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:51.251628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:51.252153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:51.252426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:51.252690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:51.254457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:51.316193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:51.317899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:51.328089 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 946/1896] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:21:51.354076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:51.355614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:51.457688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c02d1-d7ba-47ff-b00f-19a32b23a97b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:51.462536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:51.463179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:51.463455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:51.464511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] PATH_INFO: `/v3/projects/5043d10b6bc04c71bd34ce8e876596fe` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:51.464888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5043d10b6bc04c71bd34ce8e876596fe)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:51.466850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:51.579379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5043d10b6bc04c71bd34ce8e876596fe', 'request_id': 'req-84e2735a-7f05-4758-8f20-9a6e20997729'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.621680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:51.703791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5043d10b6bc04c71bd34ce8e876596fe', 'request_id': 'req-84e2735a-7f05-4758-8f20-9a6e20997729'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.710843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5043d10b6bc04c71bd34ce8e876596fe', 'request_id': 'req-84e2735a-7f05-4758-8f20-9a6e20997729'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.726858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] The token cache is being invalidate because project 5043d10b6bc04c71bd34ce8e876596fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:51.727325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.728239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:51.730202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 951/1897] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:51 2022] DELETE /identity/v3/projects/5043d10b6bc04c71bd34ce8e876596fe => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:51.734214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:51.745469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec6e89-d441-46c0-97ed-712288f65749 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:51.814641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:51.815279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:51.815550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:51.815819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] PATH_INFO: `/v3/users/c0e63327bdf4411f8c9fe4b93ef3a38b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:51.816952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e63327bdf4411f8c9fe4b93ef3a38b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:51.819132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:51.850804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:51.980277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:51.983434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:52.039748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e63327bdf4411f8c9fe4b93ef3a38b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.043126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2735a-7f05-4758-8f20-9a6e20997729 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:52.068262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:52.068916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:52.069189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:52.069455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] PATH_INFO: `/v3/projects/a5bb6eff9887465999946e8e5fb0712b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:52.071001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5bb6eff9887465999946e8e5fb0712b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:52.073136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:52.110477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e63327bdf4411f8c9fe4b93ef3a38b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.111665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e63327bdf4411f8c9fe4b93ef3a38b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.121905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e63327bdf4411f8c9fe4b93ef3a38b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.152969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b3ff14c9-640b-4f1a-92a5-e46b1531a971 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e63327bdf4411f8c9fe4b93ef3a38b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.161752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 947/1898] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:51 2022] DELETE /identity/v3/users/c0e63327bdf4411f8c9fe4b93ef3a38b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:52.170975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434b0e-f835-426e-8823-190663a98ec7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:52.170975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434b0e-f835-426e-8823-190663a98ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:52.170975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da434b0e-f835-426e-8823-190663a98ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:52.190087 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-da434b0e-f835-426e-8823-190663a98ec7 None None] MFA Rules not processed for user `72467431e40c491cad192b06be494175`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:52.191412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-da434b0e-f835-426e-8823-190663a98ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:52.242328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5bb6eff9887465999946e8e5fb0712b', 'request_id': 'req-53493b69-c222-4b9f-b21b-0c92dd20e4ac'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.251492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 948/1899] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:52.331727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5bb6eff9887465999946e8e5fb0712b', 'request_id': 'req-53493b69-c222-4b9f-b21b-0c92dd20e4ac'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.339262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5bb6eff9887465999946e8e5fb0712b', 'request_id': 'req-53493b69-c222-4b9f-b21b-0c92dd20e4ac'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.352421 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] The token cache is being invalidate because project a5bb6eff9887465999946e8e5fb0712b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:52.352789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.354022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-53493b69-c222-4b9f-b21b-0c92dd20e4ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:52.356093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 952/1900] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:51 2022] DELETE /identity/v3/projects/a5bb6eff9887465999946e8e5fb0712b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:52.362211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fdc41-54a6-4c9c-b4dd-7bd5051a3018 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:52.366406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fdc41-54a6-4c9c-b4dd-7bd5051a3018 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:52.366406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fdc41-54a6-4c9c-b4dd-7bd5051a3018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:52.383849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a70fdc41-54a6-4c9c-b4dd-7bd5051a3018 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:52.385603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a70fdc41-54a6-4c9c-b4dd-7bd5051a3018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:52.480245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 949/1901] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:52.489191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff05f01-0a8d-4304-b810-27bf6c1ca9a8 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:52.489664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff05f01-0a8d-4304-b810-27bf6c1ca9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:52.490004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff05f01-0a8d-4304-b810-27bf6c1ca9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:52.520853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6ff05f01-0a8d-4304-b810-27bf6c1ca9a8 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:52.522077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6ff05f01-0a8d-4304-b810-27bf6c1ca9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:52.535795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 953/1902] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:52.540925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fdc41-54a6-4c9c-b4dd-7bd5051a3018 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:52.555973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fdc41-54a6-4c9c-b4dd-7bd5051a3018 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:52.565797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:52.566753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:52.567132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:52.568101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:52.569888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:52.569888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:52.571839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:52.601059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 950/1903] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:21:52.605836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff05f01-0a8d-4304-b810-27bf6c1ca9a8 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:52.621462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff05f01-0a8d-4304-b810-27bf6c1ca9a8 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:52.636537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:52.637565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:52.637565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:52.637898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:52.638575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:52.641248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:52.781756 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 954/1904] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:52 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:52.785931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:52.799023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef5cae-0cb2-40db-8677-34fbfad55dae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:52.890494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb946c22-577d-48c9-b691-4281508eee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:52.891158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb946c22-577d-48c9-b691-4281508eee74 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:52.891657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb946c22-577d-48c9-b691-4281508eee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:52.891988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb946c22-577d-48c9-b691-4281508eee74 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:52.892621 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb946c22-577d-48c9-b691-4281508eee74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:52.894585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb946c22-577d-48c9-b691-4281508eee74 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:52.996060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:52.996060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:52.996709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78414624-5c95-4109-a3b7-d29401978e16 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.001845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.002586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.002878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.003455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] PATH_INFO: `/v3/projects/2762430dc7824dae95bf74bf40eaf9d6` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.004754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2762430dc7824dae95bf74bf40eaf9d6)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.006889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.026854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 951/1905] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:52 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:53.028509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb946c22-577d-48c9-b691-4281508eee74 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.060284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb946c22-577d-48c9-b691-4281508eee74 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.067548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7470592-5b12-429a-bec7-84eaec01b400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.071620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7470592-5b12-429a-bec7-84eaec01b400 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.071620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7470592-5b12-429a-bec7-84eaec01b400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.071620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7470592-5b12-429a-bec7-84eaec01b400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.071620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7470592-5b12-429a-bec7-84eaec01b400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.071983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7470592-5b12-429a-bec7-84eaec01b400 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.084377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 952/1906] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:53.089577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7470592-5b12-429a-bec7-84eaec01b400 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.102334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7470592-5b12-429a-bec7-84eaec01b400 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.110856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cfd96-812c-40db-b071-09d001621157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.111841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cfd96-812c-40db-b071-09d001621157 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.112426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cfd96-812c-40db-b071-09d001621157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.112426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cfd96-812c-40db-b071-09d001621157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.113198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cfd96-812c-40db-b071-09d001621157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.114973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cfd96-812c-40db-b071-09d001621157 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.129684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 953/1907] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:53.139002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cfd96-812c-40db-b071-09d001621157 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.152340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cfd96-812c-40db-b071-09d001621157 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.160004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129790a8-8f06-45ef-a37b-51638d3f8363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.160843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129790a8-8f06-45ef-a37b-51638d3f8363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.161127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129790a8-8f06-45ef-a37b-51638d3f8363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.161629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129790a8-8f06-45ef-a37b-51638d3f8363 admin admin] PATH_INFO: `/v3/projects/d95a2cd313b64b2984a971f58d7ba34c/users/82f9913944f74c8b915b57ab4339fd47/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.190242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129790a8-8f06-45ef-a37b-51638d3f8363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95a2cd313b64b2984a971f58d7ba34c, user_id=82f9913944f74c8b915b57ab4339fd47, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.197549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129790a8-8f06-45ef-a37b-51638d3f8363 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.213223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2762430dc7824dae95bf74bf40eaf9d6', 'request_id': 'req-77aec233-c515-4345-87d5-3e7e834292e0'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:53.302514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 954/1908] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:53 2022] PUT /identity/v3/projects/d95a2cd313b64b2984a971f58d7ba34c/users/82f9913944f74c8b915b57ab4339fd47/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:53.308037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea94ea-9878-44c5-8ff3-cf3fc2da8f85 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.308409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea94ea-9878-44c5-8ff3-cf3fc2da8f85 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.308756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea94ea-9878-44c5-8ff3-cf3fc2da8f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.315384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2762430dc7824dae95bf74bf40eaf9d6', 'request_id': 'req-77aec233-c515-4345-87d5-3e7e834292e0'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:53.326311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2762430dc7824dae95bf74bf40eaf9d6', 'request_id': 'req-77aec233-c515-4345-87d5-3e7e834292e0'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:53.346029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] The token cache is being invalidate because project 2762430dc7824dae95bf74bf40eaf9d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:53.346625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:53.347555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-77aec233-c515-4345-87d5-3e7e834292e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:53.350332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 955/1909] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:52 2022] DELETE /identity/v3/projects/2762430dc7824dae95bf74bf40eaf9d6 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:53.355222 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6df23-518c-4c1f-9239-ebca667ecf38 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.355603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6df23-518c-4c1f-9239-ebca667ecf38 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.355741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-dfea94ea-9878-44c5-8ff3-cf3fc2da8f85 None None] MFA Rules not processed for user `82f9913944f74c8b915b57ab4339fd47`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:53.356071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6df23-518c-4c1f-9239-ebca667ecf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.359127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dfea94ea-9878-44c5-8ff3-cf3fc2da8f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:53.381338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f3d6df23-518c-4c1f-9239-ebca667ecf38 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:53.383722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f3d6df23-518c-4c1f-9239-ebca667ecf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:53.472064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 955/1910] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:53.476470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea94ea-9878-44c5-8ff3-cf3fc2da8f85 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.477914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dfea94ea-9878-44c5-8ff3-cf3fc2da8f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:53.482267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 956/1911] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:53.492857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3e509-c4bc-4bd1-bffb-5b2570910c9d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.493838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea94ea-9878-44c5-8ff3-cf3fc2da8f85 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.494010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3e509-c4bc-4bd1-bffb-5b2570910c9d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.494010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3e509-c4bc-4bd1-bffb-5b2570910c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.498102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ce984-9d73-4fdb-8c0f-d7ab5fe4f02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.498718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ce984-9d73-4fdb-8c0f-d7ab5fe4f02b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.498995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ce984-9d73-4fdb-8c0f-d7ab5fe4f02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.499264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ce984-9d73-4fdb-8c0f-d7ab5fe4f02b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.500090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ce984-9d73-4fdb-8c0f-d7ab5fe4f02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.503006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ce984-9d73-4fdb-8c0f-d7ab5fe4f02b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.515810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-8be3e509-c4bc-4bd1-bffb-5b2570910c9d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:53.517034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8be3e509-c4bc-4bd1-bffb-5b2570910c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:53.525280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 957/1912] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:53.529328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3e509-c4bc-4bd1-bffb-5b2570910c9d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.538412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3e509-c4bc-4bd1-bffb-5b2570910c9d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.543433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.544026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.544302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.544572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.545213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:21:53.545513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.547820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.559643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 958/1913] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:21:53.565212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.576309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de1640-c5ad-43ff-977c-9906836209cd admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.581296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e52f7-33f5-4d95-82a1-d0f3e3279bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.581952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e52f7-33f5-4d95-82a1-d0f3e3279bad admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.582248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e52f7-33f5-4d95-82a1-d0f3e3279bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.582549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e52f7-33f5-4d95-82a1-d0f3e3279bad admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.583188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e52f7-33f5-4d95-82a1-d0f3e3279bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.585115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e52f7-33f5-4d95-82a1-d0f3e3279bad admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.605059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:53.607930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 956/1914] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:53 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:53.611986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ce984-9d73-4fdb-8c0f-d7ab5fe4f02b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.677280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ce984-9d73-4fdb-8c0f-d7ab5fe4f02b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.682705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedeb13d-3d13-4969-bdf5-013ed2c2764b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.683549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeb13d-3d13-4969-bdf5-013ed2c2764b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.689174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeb13d-3d13-4969-bdf5-013ed2c2764b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.689174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedeb13d-3d13-4969-bdf5-013ed2c2764b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.689174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedeb13d-3d13-4969-bdf5-013ed2c2764b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.689174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedeb13d-3d13-4969-bdf5-013ed2c2764b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.693282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 959/1915] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:53 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:53.694534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e52f7-33f5-4d95-82a1-d0f3e3279bad admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.784592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 957/1916] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:53 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:53.785218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e52f7-33f5-4d95-82a1-d0f3e3279bad admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.789970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedeb13d-3d13-4969-bdf5-013ed2c2764b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.790616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d3635-df09-4700-90bf-391f1a3c5a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.791351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d3635-df09-4700-90bf-391f1a3c5a63 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.791696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d3635-df09-4700-90bf-391f1a3c5a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.792069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d3635-df09-4700-90bf-391f1a3c5a63 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.792836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d3635-df09-4700-90bf-391f1a3c5a63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.794838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d3635-df09-4700-90bf-391f1a3c5a63 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.801895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedeb13d-3d13-4969-bdf5-013ed2c2764b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.807659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576c8d-df43-4521-ad40-cb25f0911c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.808542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576c8d-df43-4521-ad40-cb25f0911c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.808890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576c8d-df43-4521-ad40-cb25f0911c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.809843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576c8d-df43-4521-ad40-cb25f0911c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.809962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0576c8d-df43-4521-ad40-cb25f0911c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.812081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0576c8d-df43-4521-ad40-cb25f0911c00 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.824996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 958/1917] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:21:53.829846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576c8d-df43-4521-ad40-cb25f0911c00 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.851759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576c8d-df43-4521-ad40-cb25f0911c00 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.863018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfbadb-e3d8-4944-a842-dd8695257d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.863616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfbadb-e3d8-4944-a842-dd8695257d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.863893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfbadb-e3d8-4944-a842-dd8695257d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.864162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfbadb-e3d8-4944-a842-dd8695257d10 admin admin] PATH_INFO: `/v3/projects/754c9d28d13144fb905b71f27ccc7594/users/49b28a7cd7c84a1ea917482cf7614e45/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.881165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfbadb-e3d8-4944-a842-dd8695257d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=754c9d28d13144fb905b71f27ccc7594, user_id=49b28a7cd7c84a1ea917482cf7614e45, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.883978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfbadb-e3d8-4944-a842-dd8695257d10 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.908008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 960/1918] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:53 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:53.913022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d3635-df09-4700-90bf-391f1a3c5a63 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.922049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d3635-df09-4700-90bf-391f1a3c5a63 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.927561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b132c9-61d2-4115-b676-cab8b22ca2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.928163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b132c9-61d2-4115-b676-cab8b22ca2ac admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.928442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b132c9-61d2-4115-b676-cab8b22ca2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.928716 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b132c9-61d2-4115-b676-cab8b22ca2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.929356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b132c9-61d2-4115-b676-cab8b22ca2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.931362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b132c9-61d2-4115-b676-cab8b22ca2ac admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.941920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 961/1919] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:21:53.945963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b132c9-61d2-4115-b676-cab8b22ca2ac admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.957423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b132c9-61d2-4115-b676-cab8b22ca2ac admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:53.964925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d4ca6-4e27-48f6-8259-e8da57bba72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:53.965786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4ca6-4e27-48f6-8259-e8da57bba72a admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:53.966245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4ca6-4e27-48f6-8259-e8da57bba72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:53.966713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4ca6-4e27-48f6-8259-e8da57bba72a admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:53.967599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d4ca6-4e27-48f6-8259-e8da57bba72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:53.971309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d4ca6-4e27-48f6-8259-e8da57bba72a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:53.979525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 959/1920] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:53 2022] PUT /identity/v3/projects/754c9d28d13144fb905b71f27ccc7594/users/49b28a7cd7c84a1ea917482cf7614e45/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:21:53.983349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfbadb-e3d8-4944-a842-dd8695257d10 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:53.989539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 962/1921] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:21:53.994139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d4ca6-4e27-48f6-8259-e8da57bba72a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:54.006975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d4ca6-4e27-48f6-8259-e8da57bba72a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:54.067026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfbadb-e3d8-4944-a842-dd8695257d10 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:54.072167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a97023-628e-4ec1-a9ab-b51be166cd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:54.074146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a97023-628e-4ec1-a9ab-b51be166cd4f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.074146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a97023-628e-4ec1-a9ab-b51be166cd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.074146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a97023-628e-4ec1-a9ab-b51be166cd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.074146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a97023-628e-4ec1-a9ab-b51be166cd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:54.076352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a97023-628e-4ec1-a9ab-b51be166cd4f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:54.091562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 960/1922] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:54.091875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3aac94-c803-4722-8b04-662266666525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:54.092391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aac94-c803-4722-8b04-662266666525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.092650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aac94-c803-4722-8b04-662266666525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.092948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3aac94-c803-4722-8b04-662266666525 admin admin] PATH_INFO: `/v3/projects/eab03443e1be4c52a935b294c6f6a9e1/users/7c62c8c442c34f1f84d56f365c847e60/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.096525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a97023-628e-4ec1-a9ab-b51be166cd4f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:54.107996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a97023-628e-4ec1-a9ab-b51be166cd4f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:54.113385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3aac94-c803-4722-8b04-662266666525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab03443e1be4c52a935b294c6f6a9e1, user_id=7c62c8c442c34f1f84d56f365c847e60, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:54.113763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c08776-0386-4e6f-b8d2-f927f07e96ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:54.114971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c08776-0386-4e6f-b8d2-f927f07e96ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.115349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c08776-0386-4e6f-b8d2-f927f07e96ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.115787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c08776-0386-4e6f-b8d2-f927f07e96ba admin admin] PATH_INFO: `/v3/projects/754c9d28d13144fb905b71f27ccc7594/users/49b28a7cd7c84a1ea917482cf7614e45/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.116188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3aac94-c803-4722-8b04-662266666525 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:54.118088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c08776-0386-4e6f-b8d2-f927f07e96ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=754c9d28d13144fb905b71f27ccc7594, user_id=49b28a7cd7c84a1ea917482cf7614e45, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:54.121084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c08776-0386-4e6f-b8d2-f927f07e96ba admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:54.136019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 961/1923] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:54 2022] PUT /identity/v3/projects/754c9d28d13144fb905b71f27ccc7594/users/49b28a7cd7c84a1ea917482cf7614e45/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:54.153547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d0c1a-9ac7-4e85-b093-f2144e816e1e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.154012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d0c1a-9ac7-4e85-b093-f2144e816e1e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.154454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d0c1a-9ac7-4e85-b093-f2144e816e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.190195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6e5d0c1a-9ac7-4e85-b093-f2144e816e1e None None] MFA Rules not processed for user `49b28a7cd7c84a1ea917482cf7614e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:54.200884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6e5d0c1a-9ac7-4e85-b093-f2144e816e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:54.271327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 963/1924] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:53 2022] PUT /identity/v3/projects/eab03443e1be4c52a935b294c6f6a9e1/users/7c62c8c442c34f1f84d56f365c847e60/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:54.281234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01098a1b-a5a7-4d28-ab54-51707befbeb6 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.282449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01098a1b-a5a7-4d28-ab54-51707befbeb6 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.282928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01098a1b-a5a7-4d28-ab54-51707befbeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.353287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 962/1925] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:54.377296 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d0c1a-9ac7-4e85-b093-f2144e816e1e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:54.380536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6e5d0c1a-9ac7-4e85-b093-f2144e816e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:54.397758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-01098a1b-a5a7-4d28-ab54-51707befbeb6 None None] MFA Rules not processed for user `7c62c8c442c34f1f84d56f365c847e60`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:54.401673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-01098a1b-a5a7-4d28-ab54-51707befbeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:54.502081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d0c1a-9ac7-4e85-b093-f2144e816e1e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:54.534005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 964/1926] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:54.539133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01098a1b-a5a7-4d28-ab54-51707befbeb6 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:54.572661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12737997-e6cd-4542-8e0e-e00d0d193dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:54.574400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12737997-e6cd-4542-8e0e-e00d0d193dfd service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.574906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12737997-e6cd-4542-8e0e-e00d0d193dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.575346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12737997-e6cd-4542-8e0e-e00d0d193dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.628769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01098a1b-a5a7-4d28-ab54-51707befbeb6 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:54.633951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebb87e-f152-40a5-b5bd-9a69b1caff9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:54.635479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebb87e-f152-40a5-b5bd-9a69b1caff9d admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.638736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebb87e-f152-40a5-b5bd-9a69b1caff9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.639018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebb87e-f152-40a5-b5bd-9a69b1caff9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.639733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebb87e-f152-40a5-b5bd-9a69b1caff9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:54.642990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebb87e-f152-40a5-b5bd-9a69b1caff9d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:54.667081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12737997-e6cd-4542-8e0e-e00d0d193dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:54.669350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12737997-e6cd-4542-8e0e-e00d0d193dfd service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:54.680048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 963/1927] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:54.779951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 965/1928] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:21:54 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:21:54.781016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12737997-e6cd-4542-8e0e-e00d0d193dfd service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:54.798434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12737997-e6cd-4542-8e0e-e00d0d193dfd service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:54.864210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74357433-439f-437f-b3d4-a3ff72d13d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:54.864210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74357433-439f-437f-b3d4-a3ff72d13d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.864210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74357433-439f-437f-b3d4-a3ff72d13d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.864210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74357433-439f-437f-b3d4-a3ff72d13d7f admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.864860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74357433-439f-437f-b3d4-a3ff72d13d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:54.867036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74357433-439f-437f-b3d4-a3ff72d13d7f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:54.958475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 964/1929] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:21:54 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:21:54.960259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebb87e-f152-40a5-b5bd-9a69b1caff9d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:54.971784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebb87e-f152-40a5-b5bd-9a69b1caff9d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:54.976934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181964-7229-4a68-b719-c4fa091005dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:54.977566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181964-7229-4a68-b719-c4fa091005dc admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:54.977845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181964-7229-4a68-b719-c4fa091005dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:54.978147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181964-7229-4a68-b719-c4fa091005dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:54.979156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58181964-7229-4a68-b719-c4fa091005dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:54.980776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58181964-7229-4a68-b719-c4fa091005dc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:54.992162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 966/1930] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:55.001115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74357433-439f-437f-b3d4-a3ff72d13d7f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:55.009867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74357433-439f-437f-b3d4-a3ff72d13d7f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:55.014705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cad4d2-5c8b-4e8f-8ed1-aaa4aa4040cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:55.015439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cad4d2-5c8b-4e8f-8ed1-aaa4aa4040cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:55.015830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cad4d2-5c8b-4e8f-8ed1-aaa4aa4040cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:55.016201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cad4d2-5c8b-4e8f-8ed1-aaa4aa4040cc admin admin] PATH_INFO: `/v3/projects/33c1c858cbfa4412b406436923af2b30/users/aacf2063515947e3ba66867a422bf76b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:55.031351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cad4d2-5c8b-4e8f-8ed1-aaa4aa4040cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c1c858cbfa4412b406436923af2b30, user_id=aacf2063515947e3ba66867a422bf76b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:55.034079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cad4d2-5c8b-4e8f-8ed1-aaa4aa4040cc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:55.102088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:55.108599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 965/1931] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:54 2022] PUT /identity/v3/projects/33c1c858cbfa4412b406436923af2b30/users/aacf2063515947e3ba66867a422bf76b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:55.118984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181964-7229-4a68-b719-c4fa091005dc admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:55.128751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58181964-7229-4a68-b719-c4fa091005dc admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:55.144352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cad4d2-5c8b-4e8f-8ed1-aaa4aa4040cc admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:55.210814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cad4d2-5c8b-4e8f-8ed1-aaa4aa4040cc admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:55.213753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac34a21-7f20-49ed-b5a1-5a0543f6279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:55.214408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac34a21-7f20-49ed-b5a1-5a0543f6279c admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:55.214647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:55.214765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac34a21-7f20-49ed-b5a1-5a0543f6279c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:55.214998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac34a21-7f20-49ed-b5a1-5a0543f6279c admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:55.215252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:55.215516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:55.215623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac34a21-7f20-49ed-b5a1-5a0543f6279c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:55.215786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:55.217260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac34a21-7f20-49ed-b5a1-5a0543f6279c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:55.237338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 967/1932] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:21:55.237338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac34a21-7f20-49ed-b5a1-5a0543f6279c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:55.248133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac34a21-7f20-49ed-b5a1-5a0543f6279c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:55.265707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc284932-5c5e-479a-b93d-849b780c12dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:55.265707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc284932-5c5e-479a-b93d-849b780c12dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:55.265707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc284932-5c5e-479a-b93d-849b780c12dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:55.265707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc284932-5c5e-479a-b93d-849b780c12dd admin admin] PATH_INFO: `/v3/projects/33c1c858cbfa4412b406436923af2b30/users/aacf2063515947e3ba66867a422bf76b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:55.266798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc284932-5c5e-479a-b93d-849b780c12dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=33c1c858cbfa4412b406436923af2b30, user_id=aacf2063515947e3ba66867a422bf76b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:55.266798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc284932-5c5e-479a-b93d-849b780c12dd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:55.271199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 968/1933] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:21:55 2022] PUT /identity/v3/projects/33c1c858cbfa4412b406436923af2b30/users/aacf2063515947e3ba66867a422bf76b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:55.281052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f615f7f-8259-442f-803d-42c9ac0e06ae None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:55.281890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f615f7f-8259-442f-803d-42c9ac0e06ae None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:55.282394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f615f7f-8259-442f-803d-42c9ac0e06ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:55.289718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:55.289718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:55.296912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 966/1934] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:55.311332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2f615f7f-8259-442f-803d-42c9ac0e06ae None None] MFA Rules not processed for user `aacf2063515947e3ba66867a422bf76b`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:55.313934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2f615f7f-8259-442f-803d-42c9ac0e06ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:55.400719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 969/1935] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:21:55.615548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:55.617407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:55.628107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b08dbf-ec87-4dbd-a415-9779e7858758 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:55.692891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70cd61-807e-4a1d-aca1-dfa566b56fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:55.693507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70cd61-807e-4a1d-aca1-dfa566b56fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:55.693956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70cd61-807e-4a1d-aca1-dfa566b56fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:55.694147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70cd61-807e-4a1d-aca1-dfa566b56fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:55.703979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70cd61-807e-4a1d-aca1-dfa566b56fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:55.705558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70cd61-807e-4a1d-aca1-dfa566b56fd4 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:55.715418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 967/1936] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:56.008334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:56.169311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f615f7f-8259-442f-803d-42c9ac0e06ae None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:56.171435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2f615f7f-8259-442f-803d-42c9ac0e06ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:56.252991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f615f7f-8259-442f-803d-42c9ac0e06ae None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:56.260991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:56.262124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:56.262599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:56.262934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] PATH_INFO: `/v3/users/87bda7c87902448693ae2169631db9fd` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:56.264240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87bda7c87902448693ae2169631db9fd)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:56.266525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:56.483968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87bda7c87902448693ae2169631db9fd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:56.485265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87bda7c87902448693ae2169631db9fd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:56.566441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87bda7c87902448693ae2169631db9fd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:56.596353 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87bda7c87902448693ae2169631db9fd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:56.605066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87bda7c87902448693ae2169631db9fd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:56.614017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 970/1937] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:56 2022] DELETE /identity/v3/users/87bda7c87902448693ae2169631db9fd => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:56.618698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26dbf1-68ae-45fa-b317-770917c5087f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:56.619194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26dbf1-68ae-45fa-b317-770917c5087f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:56.619306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26dbf1-68ae-45fa-b317-770917c5087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:56.639551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2e26dbf1-68ae-45fa-b317-770917c5087f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:56.641669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2e26dbf1-68ae-45fa-b317-770917c5087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:56.715030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 968/1938] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:56.725654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:56.727157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:56.797951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767f542-6a12-4ae8-8f19-8f026980fb68 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:56.801715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:56.807401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:56.807401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:56.807401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:56.813428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:56.814888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:56.825057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 971/1939] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:57.223020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26dbf1-68ae-45fa-b317-770917c5087f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:57.235594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2e26dbf1-68ae-45fa-b317-770917c5087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:57.255846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26dbf1-68ae-45fa-b317-770917c5087f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:57.269823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:57.270630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:57.270908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:57.271236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] PATH_INFO: `/v3/users/7c62c8c442c34f1f84d56f365c847e60` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:57.273091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c62c8c442c34f1f84d56f365c847e60)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:57.275690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:57.543904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c62c8c442c34f1f84d56f365c847e60'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:57.664605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c62c8c442c34f1f84d56f365c847e60'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:57.664605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c62c8c442c34f1f84d56f365c847e60'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:57.673654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c62c8c442c34f1f84d56f365c847e60'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:57.682722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:57.684234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:57.739171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d5334137-74c9-485f-a376-fe01aa48f6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c62c8c442c34f1f84d56f365c847e60'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:57.745005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5884460-f3d5-4f0d-9e19-96872031966c service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:57.746549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 969/1940] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:57 2022] DELETE /identity/v3/users/7c62c8c442c34f1f84d56f365c847e60 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:57.752695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cde23-9b84-4099-b04e-006c99797c51 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:57.753251 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cde23-9b84-4099-b04e-006c99797c51 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:57.753737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cde23-9b84-4099-b04e-006c99797c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:57.785468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-098cde23-9b84-4099-b04e-006c99797c51 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:21:57.790271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-098cde23-9b84-4099-b04e-006c99797c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:57.869503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:57.872021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:57.872477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:57.872753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] PATH_INFO: `/v3/projects/e6d856552078444d9c2382a1f29c5b3b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:57.874998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d856552078444d9c2382a1f29c5b3b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:57.880360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:57.927944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 970/1941] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:21:57.957119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cde23-9b84-4099-b04e-006c99797c51 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:57.959957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-098cde23-9b84-4099-b04e-006c99797c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:58.035058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d856552078444d9c2382a1f29c5b3b', 'request_id': 'req-9abf58c5-98c3-40e2-abc9-449ff839a34d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.067194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cde23-9b84-4099-b04e-006c99797c51 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:58.088155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:58.118434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d856552078444d9c2382a1f29c5b3b', 'request_id': 'req-9abf58c5-98c3-40e2-abc9-449ff839a34d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.126077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d856552078444d9c2382a1f29c5b3b', 'request_id': 'req-9abf58c5-98c3-40e2-abc9-449ff839a34d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.136438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:58.137386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:58.137846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:58.138372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:58.141329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] The token cache is being invalidate because project e6d856552078444d9c2382a1f29c5b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:21:58.141714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.142747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.147403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 972/1942] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:57 2022] DELETE /identity/v3/projects/e6d856552078444d9c2382a1f29c5b3b => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:58.153141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:58.160577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:58.246609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:58.251080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:58.273870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 971/1943] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:21:58.278216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf58c5-98c3-40e2-abc9-449ff839a34d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:58.286670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:58.286670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:58.286670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:58.286670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] PATH_INFO: `/v3/users/3a4d25eedb3e4b328954295d00c94184` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:58.288451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a4d25eedb3e4b328954295d00c94184)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:58.290010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:58.582111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a4d25eedb3e4b328954295d00c94184'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.582785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a4d25eedb3e4b328954295d00c94184'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.652262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:58.708842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a4d25eedb3e4b328954295d00c94184'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.733245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a4d25eedb3e4b328954295d00c94184'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.741112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a4d25eedb3e4b328954295d00c94184'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:21:58.749549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 973/1944] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:21:58 2022] DELETE /identity/v3/users/3a4d25eedb3e4b328954295d00c94184 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:21:59.433144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:21:59.743287 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:59.745378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:59.762659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b24567-d22d-4167-81b0-90b4b6e1edf3 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:59.927577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:59.928238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:59.928561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:59.928890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] PATH_INFO: `/v3/projects/eab03443e1be4c52a935b294c6f6a9e1` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:59.935320 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab03443e1be4c52a935b294c6f6a9e1)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:59.937262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:21:59.939702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:21:59.941187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:21:59.958583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4306b-5ceb-458f-b61b-37d7d2a7c826 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:21:59.983917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:21:59.985806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:21:59.989084 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:21:59.989084 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] PATH_INFO: `/v3/projects/a0101fde4d0c4ddb8a7a70f09e313618` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:21:59.989084 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0101fde4d0c4ddb8a7a70f09e313618)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:21:59.991498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:00.127209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0101fde4d0c4ddb8a7a70f09e313618', 'request_id': 'req-7dff36a5-cd86-4a18-a856-df2c07ec9511'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.143857 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab03443e1be4c52a935b294c6f6a9e1', 'request_id': 'req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.195450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0101fde4d0c4ddb8a7a70f09e313618', 'request_id': 'req-7dff36a5-cd86-4a18-a856-df2c07ec9511'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.201831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0101fde4d0c4ddb8a7a70f09e313618', 'request_id': 'req-7dff36a5-cd86-4a18-a856-df2c07ec9511'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.229617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] The token cache is being invalidate because project a0101fde4d0c4ddb8a7a70f09e313618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:00.230031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.230864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-7dff36a5-cd86-4a18-a856-df2c07ec9511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.236243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 974/1945] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:59 2022] DELETE /identity/v3/projects/a0101fde4d0c4ddb8a7a70f09e313618 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:00.237827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8285e-0e40-4acd-96e5-f4de0d9a5473 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:00.238113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8285e-0e40-4acd-96e5-f4de0d9a5473 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:00.239653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8285e-0e40-4acd-96e5-f4de0d9a5473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:00.255471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab03443e1be4c52a935b294c6f6a9e1', 'request_id': 'req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.262291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab03443e1be4c52a935b294c6f6a9e1', 'request_id': 'req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.294844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] The token cache is being invalidate because project eab03443e1be4c52a935b294c6f6a9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:00.296188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.298519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e2a8285e-0e40-4acd-96e5-f4de0d9a5473 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:00.299013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.302725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e2a8285e-0e40-4acd-96e5-f4de0d9a5473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:00.306453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 972/1946] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:21:59 2022] DELETE /identity/v3/projects/eab03443e1be4c52a935b294c6f6a9e1 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:00.315549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:00.318105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:00.383774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:00.447119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:00.502584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 975/1947] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:00.507163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd123e-c1a0-4b6b-89bb-a9d9bf267bd5 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:00.507571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd123e-c1a0-4b6b-89bb-a9d9bf267bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:00.507943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd123e-c1a0-4b6b-89bb-a9d9bf267bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:00.522922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5e94d-dc22-4fe0-bd0f-31d8071b7a4e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:00.531834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:00.532499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:00.532782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:00.533054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] PATH_INFO: `/v3/users/aacf2063515947e3ba66867a422bf76b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:00.545160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacf2063515947e3ba66867a422bf76b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:00.548671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:00.556341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-81dd123e-c1a0-4b6b-89bb-a9d9bf267bd5 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:00.558179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-81dd123e-c1a0-4b6b-89bb-a9d9bf267bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:00.589998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 976/1948] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:00.591097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd123e-c1a0-4b6b-89bb-a9d9bf267bd5 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:00.602479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd123e-c1a0-4b6b-89bb-a9d9bf267bd5 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:00.607522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:00.608234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:00.608620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:00.608956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:00.610297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:00.610717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:00.612478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:00.640183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 977/1949] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:00.641440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:00.652759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221dcd0-d8a4-4982-83c4-24b1bbd88ece admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:00.663754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ed612-e4a0-4a14-a815-39971382d0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:00.664520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ed612-e4a0-4a14-a815-39971382d0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:00.664812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ed612-e4a0-4a14-a815-39971382d0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:00.665149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ed612-e4a0-4a14-a815-39971382d0d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:00.666170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ed612-e4a0-4a14-a815-39971382d0d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:00.668326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ed612-e4a0-4a14-a815-39971382d0d8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:00.787787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacf2063515947e3ba66867a422bf76b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.818168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 978/1950] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:00 2022] POST /identity/v3/projects => generated 363 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:00.821124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ed612-e4a0-4a14-a815-39971382d0d8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:00.867171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacf2063515947e3ba66867a422bf76b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.867949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacf2063515947e3ba66867a422bf76b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.878440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacf2063515947e3ba66867a422bf76b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.932583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacf2063515947e3ba66867a422bf76b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:00.942801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ed612-e4a0-4a14-a815-39971382d0d8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:00.943473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 973/1951] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:00 2022] DELETE /identity/v3/users/aacf2063515947e3ba66867a422bf76b => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:01.029975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08499d54-880e-4f37-bb25-4b306df568b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:01.031076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08499d54-880e-4f37-bb25-4b306df568b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:01.031688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08499d54-880e-4f37-bb25-4b306df568b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:01.032414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08499d54-880e-4f37-bb25-4b306df568b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:01.033348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08499d54-880e-4f37-bb25-4b306df568b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:01.035532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08499d54-880e-4f37-bb25-4b306df568b7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:01.149660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 979/1952] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:00 2022] POST /identity/v3/users => generated 282 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:01.153457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:01.154898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:01.171363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e8996-661e-4563-899b-86d8e526a968 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:01.183178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee09de-85e9-476c-8187-52f3e740821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:01.183710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee09de-85e9-476c-8187-52f3e740821f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:01.186404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee09de-85e9-476c-8187-52f3e740821f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:01.186404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee09de-85e9-476c-8187-52f3e740821f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:01.186404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee09de-85e9-476c-8187-52f3e740821f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:01.186668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee09de-85e9-476c-8187-52f3e740821f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:01.204250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 974/1953] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:01.205765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08499d54-880e-4f37-bb25-4b306df568b7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:01.216641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08499d54-880e-4f37-bb25-4b306df568b7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:01.222441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a7ab6-1a6e-4f39-b4c1-ed79a51ff57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:01.226446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a7ab6-1a6e-4f39-b4c1-ed79a51ff57c admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:01.226446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a7ab6-1a6e-4f39-b4c1-ed79a51ff57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:01.226446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a7ab6-1a6e-4f39-b4c1-ed79a51ff57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:01.226446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a7ab6-1a6e-4f39-b4c1-ed79a51ff57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:01.227513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a7ab6-1a6e-4f39-b4c1-ed79a51ff57c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:01.245136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 980/1954] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:01.246833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee09de-85e9-476c-8187-52f3e740821f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:01.278973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee09de-85e9-476c-8187-52f3e740821f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:01.289860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:01.290573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:01.290849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:01.291123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] PATH_INFO: `/v3/projects/174b56d368c448169c6cb1f55bca6216/users/51de5d4ad11d487da5c1dc08e3a6b000/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:01.310192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=174b56d368c448169c6cb1f55bca6216, user_id=51de5d4ad11d487da5c1dc08e3a6b000, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:01.320538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:01.435049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 975/1955] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:01 2022] PUT /identity/v3/projects/174b56d368c448169c6cb1f55bca6216/users/51de5d4ad11d487da5c1dc08e3a6b000/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:01.441609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ef77-307f-490e-83ba-a4535375c26b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:01.441915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ef77-307f-490e-83ba-a4535375c26b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:01.442787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355ef77-307f-490e-83ba-a4535375c26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:01.483189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2355ef77-307f-490e-83ba-a4535375c26b None None] MFA Rules not processed for user `51de5d4ad11d487da5c1dc08e3a6b000`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:01.484797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2355ef77-307f-490e-83ba-a4535375c26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:01.632597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 981/1956] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:01.633860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:01.635941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:01.636697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ef77-307f-490e-83ba-a4535375c26b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:01.727511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501af3e-5889-4d11-b4fb-766da1583adf admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:01.732760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:01.732969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355ef77-307f-490e-83ba-a4535375c26b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:01.734009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:01.734009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:01.734920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] PATH_INFO: `/v3/projects/33c1c858cbfa4412b406436923af2b30` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:01.736167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33c1c858cbfa4412b406436923af2b30)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:01.738269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:01.740348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b3b61-ab92-4721-aeb1-b16824de1f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:01.740348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b3b61-ab92-4721-aeb1-b16824de1f8d admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:01.740549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b3b61-ab92-4721-aeb1-b16824de1f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:01.740746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2b3b61-ab92-4721-aeb1-b16824de1f8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:01.741732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b3b61-ab92-4721-aeb1-b16824de1f8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:01.744412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2b3b61-ab92-4721-aeb1-b16824de1f8d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:01.878928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 982/1957] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:01 2022] POST /identity/v3/projects => generated 363 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:01.882900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b3b61-ab92-4721-aeb1-b16824de1f8d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:01.894466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c1c858cbfa4412b406436923af2b30', 'request_id': 'req-fca55866-f8c2-46a3-bd3f-089958f44688'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:01.965229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2b3b61-ab92-4721-aeb1-b16824de1f8d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:01.971154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:01.971472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:01.971748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:01.972016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:01.972678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:01.975224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:01.978221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c1c858cbfa4412b406436923af2b30', 'request_id': 'req-fca55866-f8c2-46a3-bd3f-089958f44688'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:01.988176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c1c858cbfa4412b406436923af2b30', 'request_id': 'req-fca55866-f8c2-46a3-bd3f-089958f44688'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:02.000448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] The token cache is being invalidate because project 33c1c858cbfa4412b406436923af2b30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:02.001129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:02.001798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fca55866-f8c2-46a3-bd3f-089958f44688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:02.003347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 976/1958] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:01 2022] DELETE /identity/v3/projects/33c1c858cbfa4412b406436923af2b30 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:02.010616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67025101-9b08-47cc-a022-60de52be9801 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.010896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67025101-9b08-47cc-a022-60de52be9801 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.011934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67025101-9b08-47cc-a022-60de52be9801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.033043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-67025101-9b08-47cc-a022-60de52be9801 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:02.034286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-67025101-9b08-47cc-a022-60de52be9801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:02.045804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 977/1959] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:02.053747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fdbff-ba56-45c0-a2a9-70ac85358334 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.053747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fdbff-ba56-45c0-a2a9-70ac85358334 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.053747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fdbff-ba56-45c0-a2a9-70ac85358334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.088192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:02.088192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bc3fdbff-ba56-45c0-a2a9-70ac85358334 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:02.088192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bc3fdbff-ba56-45c0-a2a9-70ac85358334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:02.091526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 983/1960] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:01 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:02.097265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 978/1961] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:02.101674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fdbff-ba56-45c0-a2a9-70ac85358334 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.104737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.106716 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:02.115442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fdbff-ba56-45c0-a2a9-70ac85358334 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.121402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.122071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.122475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.123006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.123641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93551ac1-c1ff-4c23-b443-1daf0fa743f8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.123933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:02.124237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.126527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.130536 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ee8c7-5925-43a4-af11-c9771b5e3d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.131241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee8c7-5925-43a4-af11-c9771b5e3d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.131701 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee8c7-5925-43a4-af11-c9771b5e3d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.132051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ee8c7-5925-43a4-af11-c9771b5e3d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.132759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ee8c7-5925-43a4-af11-c9771b5e3d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.135037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ee8c7-5925-43a4-af11-c9771b5e3d0c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.145072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 979/1962] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:22:02.147325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 984/1963] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:02.152254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.153369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ee8c7-5925-43a4-af11-c9771b5e3d0c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.165016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e55ed-7754-4eb3-935a-aa6854b1c57b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.166377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ee8c7-5925-43a4-af11-c9771b5e3d0c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.170129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24e2a6-9bb4-4bd2-af70-b4e1298d247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.170946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24e2a6-9bb4-4bd2-af70-b4e1298d247e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.171375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48c79-3a3c-4e62-8320-1d501c0fe48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.172664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24e2a6-9bb4-4bd2-af70-b4e1298d247e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.172823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48c79-3a3c-4e62-8320-1d501c0fe48d admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.173037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48c79-3a3c-4e62-8320-1d501c0fe48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.173309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48c79-3a3c-4e62-8320-1d501c0fe48d admin admin] PATH_INFO: `/v3/projects/2f21454bf0d743e7a5a2514e72208699/users/04076149afa54c7db363fe1d59ccd9b5/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.173726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e24e2a6-9bb4-4bd2-af70-b4e1298d247e admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.174922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24e2a6-9bb4-4bd2-af70-b4e1298d247e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.177035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e24e2a6-9bb4-4bd2-af70-b4e1298d247e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.191130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b48c79-3a3c-4e62-8320-1d501c0fe48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f21454bf0d743e7a5a2514e72208699, user_id=04076149afa54c7db363fe1d59ccd9b5, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.194194 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b48c79-3a3c-4e62-8320-1d501c0fe48d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.204561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:02.267995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 980/1964] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:02 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:02.272890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24e2a6-9bb4-4bd2-af70-b4e1298d247e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.294433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 985/1965] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:02 2022] PUT /identity/v3/projects/2f21454bf0d743e7a5a2514e72208699/users/04076149afa54c7db363fe1d59ccd9b5/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:02.299616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48c79-3a3c-4e62-8320-1d501c0fe48d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.353513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e24e2a6-9bb4-4bd2-af70-b4e1298d247e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.379084 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48c79-3a3c-4e62-8320-1d501c0fe48d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.386870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e5dad-e9a3-4a02-8662-bcfe34e9aa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.387654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e5dad-e9a3-4a02-8662-bcfe34e9aa54 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.387939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e5dad-e9a3-4a02-8662-bcfe34e9aa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.388236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e5dad-e9a3-4a02-8662-bcfe34e9aa54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.388876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e5dad-e9a3-4a02-8662-bcfe34e9aa54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.390503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e5dad-e9a3-4a02-8662-bcfe34e9aa54 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.401484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 986/1966] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:02.405652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e5dad-e9a3-4a02-8662-bcfe34e9aa54 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.420968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e5dad-e9a3-4a02-8662-bcfe34e9aa54 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.429762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202589ea-2359-441d-8ba6-734a4e1831c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.430759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202589ea-2359-441d-8ba6-734a4e1831c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.430866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d74081-24e1-42bb-b907-779df0d96dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.430955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202589ea-2359-441d-8ba6-734a4e1831c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.431025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202589ea-2359-441d-8ba6-734a4e1831c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.431431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74081-24e1-42bb-b907-779df0d96dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.431614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202589ea-2359-441d-8ba6-734a4e1831c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.431706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74081-24e1-42bb-b907-779df0d96dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.431963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74081-24e1-42bb-b907-779df0d96dd0 admin admin] PATH_INFO: `/v3/projects/2f21454bf0d743e7a5a2514e72208699/users/04076149afa54c7db363fe1d59ccd9b5/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.433575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202589ea-2359-441d-8ba6-734a4e1831c4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.434073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d74081-24e1-42bb-b907-779df0d96dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f21454bf0d743e7a5a2514e72208699, user_id=04076149afa54c7db363fe1d59ccd9b5, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.437193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d74081-24e1-42bb-b907-779df0d96dd0 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.450025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 987/1967] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:02 2022] PUT /identity/v3/projects/2f21454bf0d743e7a5a2514e72208699/users/04076149afa54c7db363fe1d59ccd9b5/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:02.461139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9264eea-34e9-456f-9604-9e72eeb6ae4f None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.461490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9264eea-34e9-456f-9604-9e72eeb6ae4f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.461845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9264eea-34e9-456f-9604-9e72eeb6ae4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.491106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e9264eea-34e9-456f-9604-9e72eeb6ae4f None None] MFA Rules not processed for user `04076149afa54c7db363fe1d59ccd9b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:02.492337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e9264eea-34e9-456f-9604-9e72eeb6ae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:02.529984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 981/1968] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:02 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:02.534105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202589ea-2359-441d-8ba6-734a4e1831c4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.544198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202589ea-2359-441d-8ba6-734a4e1831c4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.567957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 988/1969] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:02.579385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9264eea-34e9-456f-9604-9e72eeb6ae4f None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.580801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e9264eea-34e9-456f-9604-9e72eeb6ae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:02.627551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18b748-2d87-41d1-95af-ade9b86809ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.628409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b748-2d87-41d1-95af-ade9b86809ef admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.628695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b748-2d87-41d1-95af-ade9b86809ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.628961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b748-2d87-41d1-95af-ade9b86809ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.629666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18b748-2d87-41d1-95af-ade9b86809ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.631333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18b748-2d87-41d1-95af-ade9b86809ef admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.642184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 982/1970] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:22:02.645492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9264eea-34e9-456f-9604-9e72eeb6ae4f None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.649499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c483cbc-7d1e-42b9-9c4c-5e5a79d97734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.650144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18b748-2d87-41d1-95af-ade9b86809ef admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.650247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c483cbc-7d1e-42b9-9c4c-5e5a79d97734 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.650530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c483cbc-7d1e-42b9-9c4c-5e5a79d97734 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.650837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c483cbc-7d1e-42b9-9c4c-5e5a79d97734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.661943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18b748-2d87-41d1-95af-ade9b86809ef admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.667318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d45a6d-5617-47ef-9c63-ae53d35bd50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.667829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c483cbc-7d1e-42b9-9c4c-5e5a79d97734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.668138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d45a6d-5617-47ef-9c63-ae53d35bd50d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.668523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d45a6d-5617-47ef-9c63-ae53d35bd50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.668917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d45a6d-5617-47ef-9c63-ae53d35bd50d admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.669269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c483cbc-7d1e-42b9-9c4c-5e5a79d97734 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.669756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d45a6d-5617-47ef-9c63-ae53d35bd50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.673851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d45a6d-5617-47ef-9c63-ae53d35bd50d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.679985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 989/1971] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:02.685254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 983/1972] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:22:02.689605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c483cbc-7d1e-42b9-9c4c-5e5a79d97734 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.700241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c483cbc-7d1e-42b9-9c4c-5e5a79d97734 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.705270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a0554-cac2-41b5-abdb-2d18a7c33c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.705918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0554-cac2-41b5-abdb-2d18a7c33c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.706253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0554-cac2-41b5-abdb-2d18a7c33c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.706558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a0554-cac2-41b5-abdb-2d18a7c33c76 admin admin] PATH_INFO: `/v3/projects/172c9af47b7e4294860031d5747c1a66/users/3ee0edb0233b4a499191efe0b2f9561f/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.721985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a0554-cac2-41b5-abdb-2d18a7c33c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172c9af47b7e4294860031d5747c1a66, user_id=3ee0edb0233b4a499191efe0b2f9561f, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:02.724614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a0554-cac2-41b5-abdb-2d18a7c33c76 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:02.827064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 990/1973] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:02 2022] PUT /identity/v3/projects/172c9af47b7e4294860031d5747c1a66/users/3ee0edb0233b4a499191efe0b2f9561f/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:02.837492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30c14-ccd6-4fc8-8a7c-f0f7d8e0e274 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.837775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30c14-ccd6-4fc8-8a7c-f0f7d8e0e274 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.838104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c30c14-ccd6-4fc8-8a7c-f0f7d8e0e274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.869662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f1c30c14-ccd6-4fc8-8a7c-f0f7d8e0e274 None None] MFA Rules not processed for user `3ee0edb0233b4a499191efe0b2f9561f`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:02.870911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f1c30c14-ccd6-4fc8-8a7c-f0f7d8e0e274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:02.915267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a0554-cac2-41b5-abdb-2d18a7c33c76 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.939728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 984/1974] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:02.944139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c30c14-ccd6-4fc8-8a7c-f0f7d8e0e274 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:02.985305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a0554-cac2-41b5-abdb-2d18a7c33c76 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:02.989005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:02.989606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:02.989881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:02.990159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:02.991960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:03.005608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c30c14-ccd6-4fc8-8a7c-f0f7d8e0e274 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:03.010299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bb56e-1a88-45f9-8b50-d4ee1a0caa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:03.010923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bb56e-1a88-45f9-8b50-d4ee1a0caa78 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.011199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bb56e-1a88-45f9-8b50-d4ee1a0caa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.011468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bb56e-1a88-45f9-8b50-d4ee1a0caa78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.012086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339bb56e-1a88-45f9-8b50-d4ee1a0caa78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.013982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339bb56e-1a88-45f9-8b50-d4ee1a0caa78 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:03.051606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.053041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:03.061886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 991/1975] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:03.103843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 985/1976] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:02 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:03.108486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:03.120427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf15b9-05e6-4563-afa3-28769929cfc6 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:03.162183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=80,keystone:DELETE=15,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:03.184492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60bfc2-6b24-41b5-a7d3-ea1e3a3eca47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:03.185379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60bfc2-6b24-41b5-a7d3-ea1e3a3eca47 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.185663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60bfc2-6b24-41b5-a7d3-ea1e3a3eca47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.185948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60bfc2-6b24-41b5-a7d3-ea1e3a3eca47 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.186613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60bfc2-6b24-41b5-a7d3-ea1e3a3eca47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.188846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60bfc2-6b24-41b5-a7d3-ea1e3a3eca47 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:03.312299 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 992/1977] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:03 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:03.314176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bb56e-1a88-45f9-8b50-d4ee1a0caa78 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:03.324493 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bb56e-1a88-45f9-8b50-d4ee1a0caa78 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:03.329850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b1370-8cf9-4f9a-9602-191b533e6a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:03.330476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1370-8cf9-4f9a-9602-191b533e6a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.330764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1370-8cf9-4f9a-9602-191b533e6a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.331086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1370-8cf9-4f9a-9602-191b533e6a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.331758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b1370-8cf9-4f9a-9602-191b533e6a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.333413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b1370-8cf9-4f9a-9602-191b533e6a04 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:03.345902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 986/1978] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:22:03.354941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60bfc2-6b24-41b5-a7d3-ea1e3a3eca47 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:03.365349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60bfc2-6b24-41b5-a7d3-ea1e3a3eca47 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:03.370054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16404c18-71ae-4d74-b82c-6eba14bce43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:03.370686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16404c18-71ae-4d74-b82c-6eba14bce43f admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.370966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16404c18-71ae-4d74-b82c-6eba14bce43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.371235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16404c18-71ae-4d74-b82c-6eba14bce43f admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.371895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16404c18-71ae-4d74-b82c-6eba14bce43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.373532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16404c18-71ae-4d74-b82c-6eba14bce43f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:03.384354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 993/1979] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:03.392344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b1370-8cf9-4f9a-9602-191b533e6a04 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:03.403394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b1370-8cf9-4f9a-9602-191b533e6a04 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:03.409606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:03.418218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.418218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.418218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] PATH_INFO: `/v3/projects/0718e11b149b4f8093ede261f62e4225/users/2f54fffeb9604ecd945490263c9aec9b/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.432016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0718e11b149b4f8093ede261f62e4225, user_id=2f54fffeb9604ecd945490263c9aec9b, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.434877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:03.536668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 987/1980] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:03 2022] PUT /identity/v3/projects/0718e11b149b4f8093ede261f62e4225/users/2f54fffeb9604ecd945490263c9aec9b/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:03.548606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eddc-35d0-470d-b04b-fed0d5fa5e08 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.548919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eddc-35d0-470d-b04b-fed0d5fa5e08 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.549224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eddc-35d0-470d-b04b-fed0d5fa5e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.577803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-bbc3eddc-35d0-470d-b04b-fed0d5fa5e08 None None] MFA Rules not processed for user `2f54fffeb9604ecd945490263c9aec9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:03.579206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bbc3eddc-35d0-470d-b04b-fed0d5fa5e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:03.635692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:03.637394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:03.646494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 994/1981] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:03.650856 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3eddc-35d0-470d-b04b-fed0d5fa5e08 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:03.666031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:03.727722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3eddc-35d0-470d-b04b-fed0d5fa5e08 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:03.730421 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be3f0-0b0f-4045-a264-99bc5231234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:03.731048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be3f0-0b0f-4045-a264-99bc5231234a admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.731329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be3f0-0b0f-4045-a264-99bc5231234a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.731600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38be3f0-0b0f-4045-a264-99bc5231234a admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.732242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be3f0-0b0f-4045-a264-99bc5231234a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.732442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f38693-64e7-4b5b-a529-d7b5390c6731 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:03.734248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38be3f0-0b0f-4045-a264-99bc5231234a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:03.737649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:03.738371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.738726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.739286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] PATH_INFO: `/v3/users/82f9913944f74c8b915b57ab4339fd47` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.740711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82f9913944f74c8b915b57ab4339fd47)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.742723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:03.837100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 995/1982] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:03 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:03.841289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be3f0-0b0f-4045-a264-99bc5231234a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:03.903161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38be3f0-0b0f-4045-a264-99bc5231234a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:03.908826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61503c-1c8e-4514-9157-c0b392174095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:03.909535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61503c-1c8e-4514-9157-c0b392174095 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:03.910371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61503c-1c8e-4514-9157-c0b392174095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:03.910371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61503c-1c8e-4514-9157-c0b392174095 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:03.910691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61503c-1c8e-4514-9157-c0b392174095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:03.912574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61503c-1c8e-4514-9157-c0b392174095 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:04.012593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 996/1983] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:03 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:04.013206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82f9913944f74c8b915b57ab4339fd47'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:04.017297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61503c-1c8e-4514-9157-c0b392174095 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:04.025355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61503c-1c8e-4514-9157-c0b392174095 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:04.030029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9ac0e-b3ff-411f-afc0-ab0655f26443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:04.030647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9ac0e-b3ff-411f-afc0-ab0655f26443 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:04.030922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9ac0e-b3ff-411f-afc0-ab0655f26443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:04.031184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9ac0e-b3ff-411f-afc0-ab0655f26443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:04.032402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9ac0e-b3ff-411f-afc0-ab0655f26443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:04.034039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9ac0e-b3ff-411f-afc0-ab0655f26443 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:04.044373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 997/1984] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:04.048618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9ac0e-b3ff-411f-afc0-ab0655f26443 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:04.058178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9ac0e-b3ff-411f-afc0-ab0655f26443 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:04.062944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:04.063753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:04.064030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:04.064303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] PATH_INFO: `/v3/projects/a9011a3b50f34e1da995b59d9053306b/users/6991b5602fcf43908a549d0c0ae64ab5/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:04.079451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9011a3b50f34e1da995b59d9053306b, user_id=6991b5602fcf43908a549d0c0ae64ab5, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:04.082502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:04.100787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82f9913944f74c8b915b57ab4339fd47'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:04.101398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82f9913944f74c8b915b57ab4339fd47'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:04.110035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82f9913944f74c8b915b57ab4339fd47'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:04.122028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-65dd63d4-c931-4552-8907-3a3169c35797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82f9913944f74c8b915b57ab4339fd47'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:04.128653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 988/1985] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:03 2022] DELETE /identity/v3/users/82f9913944f74c8b915b57ab4339fd47 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:04.133081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcaf9-e4e4-4f97-b686-0b6cce9f5914 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:04.133362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcaf9-e4e4-4f97-b686-0b6cce9f5914 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:04.133630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fcaf9-e4e4-4f97-b686-0b6cce9f5914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:04.153031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-647fcaf9-e4e4-4f97-b686-0b6cce9f5914 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:04.154212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-647fcaf9-e4e4-4f97-b686-0b6cce9f5914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:04.211641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 989/1986] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:04.221817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fcaf9-e4e4-4f97-b686-0b6cce9f5914 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:04.222059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 998/1987] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:04 2022] PUT /identity/v3/projects/a9011a3b50f34e1da995b59d9053306b/users/6991b5602fcf43908a549d0c0ae64ab5/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:04.223320 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-647fcaf9-e4e4-4f97-b686-0b6cce9f5914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:04.229926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:04.231697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:04.282462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fcaf9-e4e4-4f97-b686-0b6cce9f5914 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:04.301748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3d489-429d-42d2-ae68-4b05ac4f4670 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:04.306748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0c4a5-db3a-4df0-a022-0765e2b9bca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:04.307355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c4a5-db3a-4df0-a022-0765e2b9bca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:04.307633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c4a5-db3a-4df0-a022-0765e2b9bca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:04.307924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f0c4a5-db3a-4df0-a022-0765e2b9bca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:04.308557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0c4a5-db3a-4df0-a022-0765e2b9bca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:04.310145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f0c4a5-db3a-4df0-a022-0765e2b9bca6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:04.320774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 999/1988] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:04.324816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0c4a5-db3a-4df0-a022-0765e2b9bca6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:04.335117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f0c4a5-db3a-4df0-a022-0765e2b9bca6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:04.335421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:04.336025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:04.336329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:04.336600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:04.341795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b3faf-e46f-462c-ba6b-68254413bcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:04.342553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b3faf-e46f-462c-ba6b-68254413bcfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:04.342932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b3faf-e46f-462c-ba6b-68254413bcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:04.343320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b3faf-e46f-462c-ba6b-68254413bcfe admin admin] PATH_INFO: `/v3/projects/a9011a3b50f34e1da995b59d9053306b/users/6991b5602fcf43908a549d0c0ae64ab5/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:04.346106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b3faf-e46f-462c-ba6b-68254413bcfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9011a3b50f34e1da995b59d9053306b, user_id=6991b5602fcf43908a549d0c0ae64ab5, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:04.348822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b3faf-e46f-462c-ba6b-68254413bcfe admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:04.350580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:04.352255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:04.361770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1000/1989] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:04 2022] PUT /identity/v3/projects/a9011a3b50f34e1da995b59d9053306b/users/6991b5602fcf43908a549d0c0ae64ab5/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:04.365308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 990/1990] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:04.371795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e15eb-af07-4df9-995f-8c17a2708133 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:04.376407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e15eb-af07-4df9-995f-8c17a2708133 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:04.376407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e15eb-af07-4df9-995f-8c17a2708133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:04.414394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-882e15eb-af07-4df9-995f-8c17a2708133 None None] MFA Rules not processed for user `6991b5602fcf43908a549d0c0ae64ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:04.414394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-882e15eb-af07-4df9-995f-8c17a2708133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:04.508027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1001/1991] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:04.532177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:04.533916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:04.606201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f6d3f-a6c1-4f08-a369-8b6aa8783daf service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:04.684242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:04.685613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:04.686026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:04.687058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:04.689355 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:04.815775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:04.819542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:04.830605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 991/1992] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:04.934093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e15eb-af07-4df9-995f-8c17a2708133 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:04.936457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-882e15eb-af07-4df9-995f-8c17a2708133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:05.033026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e15eb-af07-4df9-995f-8c17a2708133 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:05.037547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:05.038177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:05.038510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:05.038780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] PATH_INFO: `/v3/projects/d95a2cd313b64b2984a971f58d7ba34c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:05.040724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95a2cd313b64b2984a971f58d7ba34c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:05.042724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:05.166937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95a2cd313b64b2984a971f58d7ba34c', 'request_id': 'req-bd8889de-9dd3-408e-8e24-448e17391097'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.272501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95a2cd313b64b2984a971f58d7ba34c', 'request_id': 'req-bd8889de-9dd3-408e-8e24-448e17391097'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.282061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95a2cd313b64b2984a971f58d7ba34c', 'request_id': 'req-bd8889de-9dd3-408e-8e24-448e17391097'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.292669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] The token cache is being invalidate because project d95a2cd313b64b2984a971f58d7ba34c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:05.292984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.294772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.296347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1002/1993] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:04 2022] DELETE /identity/v3/projects/d95a2cd313b64b2984a971f58d7ba34c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:05.301177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:05.311906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb9693-e260-4974-8ac1-6272c1f7627e service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:05.370028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:05.370702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:05.370979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:05.371249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] PATH_INFO: `/v3/users/49b28a7cd7c84a1ea917482cf7614e45` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:05.372374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b28a7cd7c84a1ea917482cf7614e45)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:05.374316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:05.693612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b28a7cd7c84a1ea917482cf7614e45'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.787692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b28a7cd7c84a1ea917482cf7614e45'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.789256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b28a7cd7c84a1ea917482cf7614e45'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.810172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b28a7cd7c84a1ea917482cf7614e45'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.841063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-44eb3fce-b3b7-4cbb-aaa5-13b08b924f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b28a7cd7c84a1ea917482cf7614e45'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:05.862334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 992/1994] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:05 2022] DELETE /identity/v3/users/49b28a7cd7c84a1ea917482cf7614e45 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:05.912567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:06.339975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:06.341531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:06.365904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8889de-9dd3-408e-8e24-448e17391097 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:06.441065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:06.441707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:06.441981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:06.442244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] PATH_INFO: `/v3/projects/754c9d28d13144fb905b71f27ccc7594` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:06.443438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] RBAC: Authorizing `identity:delete_project(project_id=754c9d28d13144fb905b71f27ccc7594)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:06.445333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:06.564460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:06.585212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '754c9d28d13144fb905b71f27ccc7594', 'request_id': 'req-89bb93cb-865e-4246-a65a-0a2c3c98418d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:06.638814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '754c9d28d13144fb905b71f27ccc7594', 'request_id': 'req-89bb93cb-865e-4246-a65a-0a2c3c98418d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:06.653108 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '754c9d28d13144fb905b71f27ccc7594', 'request_id': 'req-89bb93cb-865e-4246-a65a-0a2c3c98418d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:06.677786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] The token cache is being invalidate because project 754c9d28d13144fb905b71f27ccc7594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:06.678082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:06.678770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-89bb93cb-865e-4246-a65a-0a2c3c98418d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:06.680435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1003/1995] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:06 2022] DELETE /identity/v3/projects/754c9d28d13144fb905b71f27ccc7594 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:06.685414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f229-705d-4556-8243-44bff447fa04 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:06.685699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f229-705d-4556-8243-44bff447fa04 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:06.685974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f229-705d-4556-8243-44bff447fa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:06.738029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d266f229-705d-4556-8243-44bff447fa04 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:06.739572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d266f229-705d-4556-8243-44bff447fa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:06.866330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 993/1996] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:06.876186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81556-c3d0-4131-ae4c-9a8e94ad819c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:06.882443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81556-c3d0-4131-ae4c-9a8e94ad819c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:06.882443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81556-c3d0-4131-ae4c-9a8e94ad819c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:06.912119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-cea81556-c3d0-4131-ae4c-9a8e94ad819c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:06.913427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cea81556-c3d0-4131-ae4c-9a8e94ad819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:06.924773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1004/1997] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:06.933472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266f229-705d-4556-8243-44bff447fa04 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:06.944913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266f229-705d-4556-8243-44bff447fa04 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:06.955649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:06.958181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:06.958181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:06.958181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:06.958181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:06.958181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:06.959501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:06.986734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 994/1998] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:06.996552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea81556-c3d0-4131-ae4c-9a8e94ad819c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.016896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea81556-c3d0-4131-ae4c-9a8e94ad819c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.041398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.042294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.042895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.043148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.052059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.052059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.166424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1005/1999] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:06 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:07.169858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.187197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdaf06-84cc-4326-9984-01c7ef27748c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.252098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.252710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.252985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.253249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.253951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.255878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.264223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.266206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:07.332484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c22f6a-f585-4b35-ae19-b8dd6577bf8e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.336324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.337011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.337386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.337723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.339588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:07.386882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 995/2000] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:07 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:07.386882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.387660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:07.399625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.401890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.402534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f30dc-8c01-4c8b-a6c0-f2a046cfdf80 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.407846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.408752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.409030 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.409303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] PATH_INFO: `/v3/users/51de5d4ad11d487da5c1dc08e3a6b000` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.416889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] RBAC: Authorizing `identity:delete_user(user_id=51de5d4ad11d487da5c1dc08e3a6b000)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.423156 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.427731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1006/2001] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:07.429427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.450204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf8574-2f2e-4708-afed-078ecd13cee1 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.457612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b75b4-dbcd-44b9-8101-8e5ec5cef3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.459776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b75b4-dbcd-44b9-8101-8e5ec5cef3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.459776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b75b4-dbcd-44b9-8101-8e5ec5cef3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.459776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b75b4-dbcd-44b9-8101-8e5ec5cef3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.459776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b75b4-dbcd-44b9-8101-8e5ec5cef3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.461685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b75b4-dbcd-44b9-8101-8e5ec5cef3f9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.475986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1007/2002] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:07.477287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b75b4-dbcd-44b9-8101-8e5ec5cef3f9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.487389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b75b4-dbcd-44b9-8101-8e5ec5cef3f9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.492006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec8db0-6677-47ee-bd80-8a980e1e2fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.492646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8db0-6677-47ee-bd80-8a980e1e2fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.494409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8db0-6677-47ee-bd80-8a980e1e2fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.494409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec8db0-6677-47ee-bd80-8a980e1e2fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.494409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec8db0-6677-47ee-bd80-8a980e1e2fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.495406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec8db0-6677-47ee-bd80-8a980e1e2fd9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.505859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1008/2003] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:22:07.509946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec8db0-6677-47ee-bd80-8a980e1e2fd9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.520755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec8db0-6677-47ee-bd80-8a980e1e2fd9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.525679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cbaac-6b41-4f9b-87c8-ef6993978594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.526459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cbaac-6b41-4f9b-87c8-ef6993978594 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.526874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cbaac-6b41-4f9b-87c8-ef6993978594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.527240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cbaac-6b41-4f9b-87c8-ef6993978594 admin admin] PATH_INFO: `/v3/projects/1811de942cac4e45bea7a28f6257aefd/users/bad1a9e6012b463c9e1337e4f8db9102/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.543341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cbaac-6b41-4f9b-87c8-ef6993978594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1811de942cac4e45bea7a28f6257aefd, user_id=bad1a9e6012b463c9e1337e4f8db9102, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.546120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6cbaac-6b41-4f9b-87c8-ef6993978594 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.649252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1009/2004] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:07 2022] PUT /identity/v3/projects/1811de942cac4e45bea7a28f6257aefd/users/bad1a9e6012b463c9e1337e4f8db9102/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:07.660003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51de5d4ad11d487da5c1dc08e3a6b000'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:07.660838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92b7f1-c18d-480e-bdd4-cc9fe2c0e969 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.661366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92b7f1-c18d-480e-bdd4-cc9fe2c0e969 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.661741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92b7f1-c18d-480e-bdd4-cc9fe2c0e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.691051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-dd92b7f1-c18d-480e-bdd4-cc9fe2c0e969 None None] MFA Rules not processed for user `bad1a9e6012b463c9e1337e4f8db9102`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:07.692274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-dd92b7f1-c18d-480e-bdd4-cc9fe2c0e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:07.726698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51de5d4ad11d487da5c1dc08e3a6b000'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:07.727646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51de5d4ad11d487da5c1dc08e3a6b000'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:07.736890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51de5d4ad11d487da5c1dc08e3a6b000'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:07.749291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b8de7a77-0434-4458-8b4f-00bf725d790a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51de5d4ad11d487da5c1dc08e3a6b000'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:07.755490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 996/2005] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:07 2022] DELETE /identity/v3/users/51de5d4ad11d487da5c1dc08e3a6b000 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:07.759859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873414e-7ab9-4b96-a736-79e79db7da89 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.760245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873414e-7ab9-4b96-a736-79e79db7da89 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.760542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873414e-7ab9-4b96-a736-79e79db7da89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.761106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1010/2006] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:07.765339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92b7f1-c18d-480e-bdd4-cc9fe2c0e969 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.767305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-dd92b7f1-c18d-480e-bdd4-cc9fe2c0e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:07.780886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9873414e-7ab9-4b96-a736-79e79db7da89 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:07.782966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9873414e-7ab9-4b96-a736-79e79db7da89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:07.839130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92b7f1-c18d-480e-bdd4-cc9fe2c0e969 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.844350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda00d3e-0583-4ca0-8036-2598f5c5aa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.845034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00d3e-0583-4ca0-8036-2598f5c5aa8b admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.845289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 997/2007] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:07.845412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00d3e-0583-4ca0-8036-2598f5c5aa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.845649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00d3e-0583-4ca0-8036-2598f5c5aa8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.846367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda00d3e-0583-4ca0-8036-2598f5c5aa8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.848369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda00d3e-0583-4ca0-8036-2598f5c5aa8b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.864105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873414e-7ab9-4b96-a736-79e79db7da89 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.865512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9873414e-7ab9-4b96-a736-79e79db7da89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:07.924836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873414e-7ab9-4b96-a736-79e79db7da89 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:07.928529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdaa94e-5bb2-4991-b212-3c03cc55bc2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:07.929149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaa94e-5bb2-4991-b212-3c03cc55bc2b service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:07.929423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaa94e-5bb2-4991-b212-3c03cc55bc2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:07.929695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdaa94e-5bb2-4991-b212-3c03cc55bc2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:07.933415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:07.940915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdaa94e-5bb2-4991-b212-3c03cc55bc2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:07.941748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1011/2008] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:07 2022] POST /identity/v3/projects => generated 381 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:07.943267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdaa94e-5bb2-4991-b212-3c03cc55bc2b service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:07.945573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda00d3e-0583-4ca0-8036-2598f5c5aa8b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:07.956046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 998/2009] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:08.012504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda00d3e-0583-4ca0-8036-2598f5c5aa8b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:08.018240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34905494-6f70-4b52-a70d-63ac2eee03af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:08.018962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34905494-6f70-4b52-a70d-63ac2eee03af admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:08.019300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34905494-6f70-4b52-a70d-63ac2eee03af admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:08.019647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34905494-6f70-4b52-a70d-63ac2eee03af admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:08.020352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34905494-6f70-4b52-a70d-63ac2eee03af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:08.022422 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34905494-6f70-4b52-a70d-63ac2eee03af admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:08.128837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1012/2010] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:07 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:08.132538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdaa94e-5bb2-4991-b212-3c03cc55bc2b service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:08.143230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdaa94e-5bb2-4991-b212-3c03cc55bc2b service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:08.147986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b168d89-9beb-49c0-a4cb-1e8122528362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:08.148579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b168d89-9beb-49c0-a4cb-1e8122528362 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:08.148894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b168d89-9beb-49c0-a4cb-1e8122528362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:08.149167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b168d89-9beb-49c0-a4cb-1e8122528362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:08.149894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b168d89-9beb-49c0-a4cb-1e8122528362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:08.151389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b168d89-9beb-49c0-a4cb-1e8122528362 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:08.165931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 999/2011] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:08.168171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34905494-6f70-4b52-a70d-63ac2eee03af admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:08.180922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34905494-6f70-4b52-a70d-63ac2eee03af admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:08.189188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1627f8-00f0-444a-86b5-cc97d99f9baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:08.189998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1627f8-00f0-444a-86b5-cc97d99f9baf admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:08.190548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1627f8-00f0-444a-86b5-cc97d99f9baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:08.190894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1627f8-00f0-444a-86b5-cc97d99f9baf admin admin] PATH_INFO: `/v3/projects/40d5eede1ef14d3baefa612bf60bff16/users/9621e88074544a3ebd36403c2c99bccb/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:08.213259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1627f8-00f0-444a-86b5-cc97d99f9baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d5eede1ef14d3baefa612bf60bff16, user_id=9621e88074544a3ebd36403c2c99bccb, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:08.213259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1627f8-00f0-444a-86b5-cc97d99f9baf admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:08.290106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1013/2012] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:08 2022] PUT /identity/v3/projects/40d5eede1ef14d3baefa612bf60bff16/users/9621e88074544a3ebd36403c2c99bccb/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:08.293910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b168d89-9beb-49c0-a4cb-1e8122528362 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:08.310324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b168d89-9beb-49c0-a4cb-1e8122528362 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:08.372610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4add9-26ed-4170-9ec0-6ce7ed220492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:08.373244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4add9-26ed-4170-9ec0-6ce7ed220492 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:08.373517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4add9-26ed-4170-9ec0-6ce7ed220492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:08.373785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4add9-26ed-4170-9ec0-6ce7ed220492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:08.374445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4add9-26ed-4170-9ec0-6ce7ed220492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:08.376296 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4add9-26ed-4170-9ec0-6ce7ed220492 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:08.393270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1000/2013] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:22:08.398533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1627f8-00f0-444a-86b5-cc97d99f9baf admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:08.412011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1627f8-00f0-444a-86b5-cc97d99f9baf admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:08.422944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:08.424109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:08.424448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:08.424722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] PATH_INFO: `/v3/projects/40d5eede1ef14d3baefa612bf60bff16/users/9621e88074544a3ebd36403c2c99bccb/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:08.428928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d5eede1ef14d3baefa612bf60bff16, user_id=9621e88074544a3ebd36403c2c99bccb, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:08.433033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:08.452638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1014/2014] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:08 2022] PUT /identity/v3/projects/40d5eede1ef14d3baefa612bf60bff16/users/9621e88074544a3ebd36403c2c99bccb/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:08.454962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040e00-0483-4d06-863b-a48eb2dcba5c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:08.455248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040e00-0483-4d06-863b-a48eb2dcba5c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:08.455656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea040e00-0483-4d06-863b-a48eb2dcba5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:08.485504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ea040e00-0483-4d06-863b-a48eb2dcba5c None None] MFA Rules not processed for user `9621e88074544a3ebd36403c2c99bccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:08.486782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ea040e00-0483-4d06-863b-a48eb2dcba5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:08.556896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1001/2015] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:08.569092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:08.570965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:08.584299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea040e00-0483-4d06-863b-a48eb2dcba5c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:08.585698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ea040e00-0483-4d06-863b-a48eb2dcba5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:08.637168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:08.649464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246f765-bbab-4b66-a9d9-35fce3de20ba admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:08.653510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:08.654124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:08.654583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:08.654839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:08.663951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea040e00-0483-4d06-863b-a48eb2dcba5c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:08.665947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:08.667611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:08.669324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:08.669683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:08.670009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:08.670281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] PATH_INFO: `/v3/projects/174b56d368c448169c6cb1f55bca6216` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:08.675142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=174b56d368c448169c6cb1f55bca6216)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:08.675142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:08.678765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1015/2016] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:08.795749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174b56d368c448169c6cb1f55bca6216', 'request_id': 'req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:08.902015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:08.904182 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:08.921894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174b56d368c448169c6cb1f55bca6216', 'request_id': 'req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:08.932054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174b56d368c448169c6cb1f55bca6216', 'request_id': 'req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:08.940304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26354c59-dbab-45b3-bafb-7ca8719aa593 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:08.949382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] The token cache is being invalidate because project 174b56d368c448169c6cb1f55bca6216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:08.950421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:08.951128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:08.953102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1002/2017] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:08 2022] DELETE /identity/v3/projects/174b56d368c448169c6cb1f55bca6216 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:08.958629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:08.965453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:09.000834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:09.001767 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:09.002043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:09.002313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:09.004288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:09.042954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065cfcb-83c2-4cbd-8fcf-c3bdc8d5d592 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:09.049321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:09.050193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:09.050742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:09.051168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] PATH_INFO: `/v3/users/04076149afa54c7db363fe1d59ccd9b5` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:09.052435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04076149afa54c7db363fe1d59ccd9b5)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:09.054442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:09.079314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:09.083100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:09.093584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1016/2018] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:09.094697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:09.096535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:09.108239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843e533-6c0c-40d3-8020-d50290f15db1 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:09.115583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:09.116104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:09.116507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:09.117030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:09.118730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:09.197663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:09.199176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:09.210315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1017/2019] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:09.241873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04076149afa54c7db363fe1d59ccd9b5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:09.326999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04076149afa54c7db363fe1d59ccd9b5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:09.330506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04076149afa54c7db363fe1d59ccd9b5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:09.342500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04076149afa54c7db363fe1d59ccd9b5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:09.357096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2c5b2a3a-c30d-4629-b8c7-0d111a340f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04076149afa54c7db363fe1d59ccd9b5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:09.363868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1003/2020] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:08 2022] DELETE /identity/v3/users/04076149afa54c7db363fe1d59ccd9b5 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:09.675635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:09.680196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:09.691154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a61bba-fc38-4bb5-84a3-f5c9558047b2 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:09.761113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:09.762016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:09.762294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:09.762589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] PATH_INFO: `/v3/users/bad1a9e6012b463c9e1337e4f8db9102` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:09.766585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad1a9e6012b463c9e1337e4f8db9102)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:09.766585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:09.837730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2528de5-22d6-4aee-98f3-26bef5d9fdbc None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:09.838259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2528de5-22d6-4aee-98f3-26bef5d9fdbc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:09.838259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2528de5-22d6-4aee-98f3-26bef5d9fdbc None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:09.839667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1004/2021] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:22:09.850713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002767f-a3e8-4f73-9ef2-6a1ca3e1e1a7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:09.850985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002767f-a3e8-4f73-9ef2-6a1ca3e1e1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:09.851250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002767f-a3e8-4f73-9ef2-6a1ca3e1e1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:09.872505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3002767f-a3e8-4f73-9ef2-6a1ca3e1e1a7 None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:09.874541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3002767f-a3e8-4f73-9ef2-6a1ca3e1e1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:09.945141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1005/2022] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:22:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:09.959612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186fc3e-87ca-4f71-80f7-d6da325e2574 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:09.959898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186fc3e-87ca-4f71-80f7-d6da325e2574 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:09.960170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186fc3e-87ca-4f71-80f7-d6da325e2574 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:09.961532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1006/2023] 10.222.0.20 () {54 vars in 961 bytes} [Fri Sep 9 23:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 09 23:22:09.966571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701710-eb21-4c45-988c-078799575315 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:09.966852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701710-eb21-4c45-988c-078799575315 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:09.974714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b701710-eb21-4c45-988c-078799575315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:09.982873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad1a9e6012b463c9e1337e4f8db9102'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:09.983606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad1a9e6012b463c9e1337e4f8db9102'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:09.997087 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4b701710-eb21-4c45-988c-078799575315 None None] MFA Rules not processed for user `a418a2d11a234b4586cb57f1c9a4d780`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:09.998511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4b701710-eb21-4c45-988c-078799575315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.015104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1007/2024] 10.222.0.20 () {58 vars in 1058 bytes} [Fri Sep 9 23:22:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:10.019443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b701710-eb21-4c45-988c-078799575315 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:10.021194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4b701710-eb21-4c45-988c-078799575315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.037308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b701710-eb21-4c45-988c-078799575315 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:10.042003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:10.042741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.043205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.043622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] PATH_INFO: `/v3/users/b043b3de9b9f4fcfaacfed75fae18306` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.044949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b043b3de9b9f4fcfaacfed75fae18306)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:10.047013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:10.078794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad1a9e6012b463c9e1337e4f8db9102'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.108311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad1a9e6012b463c9e1337e4f8db9102'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.117829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad1a9e6012b463c9e1337e4f8db9102'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.126204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1018/2025] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:09 2022] DELETE /identity/v3/users/bad1a9e6012b463c9e1337e4f8db9102 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:10.127801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:10.202177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069ce76-cca7-4b7e-b25a-faae04a139f2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:10.208042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:10.208610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.215257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.215257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.215257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.223647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:10.225263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:10.236345 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1019/2026] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:10.237205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:10.238651 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.297750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b043b3de9b9f4fcfaacfed75fae18306'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.310764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da83252-ebb7-40d1-a08f-2c3f5878237c service placement] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:10.316322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:10.316952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.318508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.318508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] PATH_INFO: `/v3/projects/2f21454bf0d743e7a5a2514e72208699` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.319322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f21454bf0d743e7a5a2514e72208699)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:10.320474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:10.372235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b043b3de9b9f4fcfaacfed75fae18306'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.372903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b043b3de9b9f4fcfaacfed75fae18306'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.381357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b043b3de9b9f4fcfaacfed75fae18306'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.394519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-158a43f7-1f52-4b02-b0f3-14a59c0914bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b043b3de9b9f4fcfaacfed75fae18306'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.400778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1008/2027] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:10 2022] DELETE /identity/v3/users/b043b3de9b9f4fcfaacfed75fae18306 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:10.404005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfed78e-f360-4a26-9223-17011d599cb2 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.404005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfed78e-f360-4a26-9223-17011d599cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.404005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfed78e-f360-4a26-9223-17011d599cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.423972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3cfed78e-f360-4a26-9223-17011d599cb2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:10.425343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3cfed78e-f360-4a26-9223-17011d599cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.442418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f21454bf0d743e7a5a2514e72208699', 'request_id': 'req-faec5bf7-b14c-4107-8e82-e1276fb44ce7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.505270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1009/2028] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:10.515992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfed78e-f360-4a26-9223-17011d599cb2 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:10.518147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f21454bf0d743e7a5a2514e72208699', 'request_id': 'req-faec5bf7-b14c-4107-8e82-e1276fb44ce7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.528580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3cfed78e-f360-4a26-9223-17011d599cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.528821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f21454bf0d743e7a5a2514e72208699', 'request_id': 'req-faec5bf7-b14c-4107-8e82-e1276fb44ce7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.543022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] The token cache is being invalidate because project 2f21454bf0d743e7a5a2514e72208699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:10.543215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.544075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-faec5bf7-b14c-4107-8e82-e1276fb44ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:10.545762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1020/2029] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:10 2022] DELETE /identity/v3/projects/2f21454bf0d743e7a5a2514e72208699 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:10.553817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3870eb-9fdb-4710-b5fc-171886d28b70 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.554304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3870eb-9fdb-4710-b5fc-171886d28b70 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.559475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3870eb-9fdb-4710-b5fc-171886d28b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.608105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-1d3870eb-9fdb-4710-b5fc-171886d28b70 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:10.611497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1d3870eb-9fdb-4710-b5fc-171886d28b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.646382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1021/2030] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:10.650138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb5eb6-e9da-4392-85af-9f6fdcedf51d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.650824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb5eb6-e9da-4392-85af-9f6fdcedf51d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.650824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb5eb6-e9da-4392-85af-9f6fdcedf51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.659589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfed78e-f360-4a26-9223-17011d599cb2 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:10.669342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:10.671449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.671449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.671449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.676679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.696022 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5fbb5eb6-e9da-4392-85af-9f6fdcedf51d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:10.701609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5fbb5eb6-e9da-4392-85af-9f6fdcedf51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.708746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:10.710206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:10.713591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1022/2031] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:10.726853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1010/2032] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:10.727054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb5eb6-e9da-4392-85af-9f6fdcedf51d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:10.755332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb5eb6-e9da-4392-85af-9f6fdcedf51d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:10.770373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:10.772558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.772908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.774405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.774405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:10.774405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:10.776303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:10.830491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1023/2033] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:22:10.831730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:10.844609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3e1da-ddf4-404b-9c32-0b899f88c332 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:10.853300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764973-976d-4ddd-a9a8-c42e47a7e540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:10.853932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764973-976d-4ddd-a9a8-c42e47a7e540 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.854443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764973-976d-4ddd-a9a8-c42e47a7e540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.855085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764973-976d-4ddd-a9a8-c42e47a7e540 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.855796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae764973-976d-4ddd-a9a8-c42e47a7e540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:10.857823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae764973-976d-4ddd-a9a8-c42e47a7e540 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:10.926944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:10.929028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:10.960753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee3009-7516-45d4-a91a-5df17aaa418a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:10.967515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:10.968290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:10.969738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:10.970031 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] PATH_INFO: `/v3/projects/a5fbef946d6240bfa7afb17096cd3ce8` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:10.972537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fbef946d6240bfa7afb17096cd3ce8)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:10.975145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:10.989730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1011/2034] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:10 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:10.992823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764973-976d-4ddd-a9a8-c42e47a7e540 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:11.151570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fbef946d6240bfa7afb17096cd3ce8', 'request_id': 'req-520e200f-908e-4745-897c-589ba6f0b8fa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.156711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764973-976d-4ddd-a9a8-c42e47a7e540 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:11.240282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:11.240949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:11.241712 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:11.241712 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:11.242218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:11.244183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:11.289617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fbef946d6240bfa7afb17096cd3ce8', 'request_id': 'req-520e200f-908e-4745-897c-589ba6f0b8fa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.296251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fbef946d6240bfa7afb17096cd3ce8', 'request_id': 'req-520e200f-908e-4745-897c-589ba6f0b8fa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.309568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] The token cache is being invalidate because project a5fbef946d6240bfa7afb17096cd3ce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:11.309858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.310547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.312122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1024/2035] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:10 2022] DELETE /identity/v3/projects/a5fbef946d6240bfa7afb17096cd3ce8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:11.317021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:11.318470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:11.331051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e200f-908e-4745-897c-589ba6f0b8fa admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:11.335688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:11.336312 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:11.336584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:11.336850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] PATH_INFO: `/v3/users/58718ac434f64aceba4262577d157d0d` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:11.337943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=58718ac434f64aceba4262577d157d0d)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:11.339881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:11.368047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1012/2036] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:10 2022] POST /identity/v3/users => generated 284 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:11.378951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:11.383404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:11.419879 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99e751-d988-40af-b322-572fd09ab50f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:11.444172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:11.445705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:11.446794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:11.448795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:11.449682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:11.454855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:11.472976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1013/2037] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:11.474128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:11.479028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:11.520008 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f1ab9-2d16-4d6e-bbf9-45246c8048f8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:11.530336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:11.532359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:11.532769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:11.533142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] PATH_INFO: `/v3/projects/1811de942cac4e45bea7a28f6257aefd` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:11.538128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1811de942cac4e45bea7a28f6257aefd)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:11.553704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:11.568111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58718ac434f64aceba4262577d157d0d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.569509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58718ac434f64aceba4262577d157d0d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.603030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:11.686440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58718ac434f64aceba4262577d157d0d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.701188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58718ac434f64aceba4262577d157d0d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.709257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58718ac434f64aceba4262577d157d0d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.716091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1025/2038] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:11 2022] DELETE /identity/v3/users/58718ac434f64aceba4262577d157d0d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:11.717319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:11.719474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:11.765647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1811de942cac4e45bea7a28f6257aefd', 'request_id': 'req-53dfc66e-599a-4495-8f63-3f3628c6d9f4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.805960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86d750-8a92-4105-8654-832a10a12a8e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:11.835866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1811de942cac4e45bea7a28f6257aefd', 'request_id': 'req-53dfc66e-599a-4495-8f63-3f3628c6d9f4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.848670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1811de942cac4e45bea7a28f6257aefd', 'request_id': 'req-53dfc66e-599a-4495-8f63-3f3628c6d9f4'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.867152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] The token cache is being invalidate because project 1811de942cac4e45bea7a28f6257aefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:11.867711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.868553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:11.870564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1014/2039] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:11 2022] DELETE /identity/v3/projects/1811de942cac4e45bea7a28f6257aefd => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:11.874360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:11.878413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:11.881043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:11.881789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:11.882686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:11.883065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:11.883853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:11.885595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:11.892976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dfc66e-599a-4495-8f63-3f3628c6d9f4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:11.898096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:11.898671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1026/2040] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:11.899198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:11.899529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:11.899802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] PATH_INFO: `/v3/users/9621e88074544a3ebd36403c2c99bccb` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:11.900977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9621e88074544a3ebd36403c2c99bccb)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:11.902744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:11.902999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:11.904502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:11.918164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb540d6-a32e-4e41-9c6b-7cdb0ce3f91d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:11.923137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:11.923789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:11.924068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:11.924340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] PATH_INFO: `/v3/projects/55cdd5fda2fd402d9df8a438b1c7d8cc/users/6ccd91b62c2f45dcb778e50f7e8af590/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:11.941068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] RBAC: Authorizing `identity:create_grant(project_id=55cdd5fda2fd402d9df8a438b1c7d8cc, user_id=6ccd91b62c2f45dcb778e50f7e8af590, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:11.943861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:12.027391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1027/2041] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:11 2022] PUT /identity/v3/projects/55cdd5fda2fd402d9df8a438b1c7d8cc/users/6ccd91b62c2f45dcb778e50f7e8af590/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:12.028239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:12.030082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:12.095234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0988ec-e516-4ab5-9b09-4ca88c8c269f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:12.099990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:12.100618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:12.100894 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:12.101267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] PATH_INFO: `/v3/projects/cfa2980ed0544c389f1fe3a463546033` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:12.102274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfa2980ed0544c389f1fe3a463546033)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:12.104451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:12.149043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9621e88074544a3ebd36403c2c99bccb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.169827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:12.239419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfa2980ed0544c389f1fe3a463546033', 'request_id': 'req-ea456c29-f473-4201-812d-321804ed64fe'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.244105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9621e88074544a3ebd36403c2c99bccb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.244730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9621e88074544a3ebd36403c2c99bccb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.253798 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9621e88074544a3ebd36403c2c99bccb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.267133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8380878b-0018-4dd9-bfdf-45abcde21f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9621e88074544a3ebd36403c2c99bccb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.274102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1015/2042] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:11 2022] DELETE /identity/v3/users/9621e88074544a3ebd36403c2c99bccb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:12.275450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714caa-c542-4475-b0c4-cbd38714ebf4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:12.275870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714caa-c542-4475-b0c4-cbd38714ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:12.276294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77714caa-c542-4475-b0c4-cbd38714ebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:12.308347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-77714caa-c542-4475-b0c4-cbd38714ebf4 None None] MFA Rules not processed for user `6ccd91b62c2f45dcb778e50f7e8af590`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:12.309688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-77714caa-c542-4475-b0c4-cbd38714ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:12.363790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfa2980ed0544c389f1fe3a463546033', 'request_id': 'req-ea456c29-f473-4201-812d-321804ed64fe'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.370642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfa2980ed0544c389f1fe3a463546033', 'request_id': 'req-ea456c29-f473-4201-812d-321804ed64fe'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.388960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] The token cache is being invalidate because project cfa2980ed0544c389f1fe3a463546033 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:12.389254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.390013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea456c29-f473-4201-812d-321804ed64fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:12.396360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1028/2043] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:12 2022] DELETE /identity/v3/projects/cfa2980ed0544c389f1fe3a463546033 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:12.397852 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f108a3-9228-4a58-a353-91512b48af7c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:12.398140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f108a3-9228-4a58-a353-91512b48af7c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:12.398441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f108a3-9228-4a58-a353-91512b48af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:12.420100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-86f108a3-9228-4a58-a353-91512b48af7c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:12.421337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-86f108a3-9228-4a58-a353-91512b48af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:12.428295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1016/2044] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:12.432812 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77714caa-c542-4475-b0c4-cbd38714ebf4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:12.434150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-77714caa-c542-4475-b0c4-cbd38714ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:12.496598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1029/2045] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:12.504458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60914d-1395-47cd-8e2d-594e13ff1fb6 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:12.505294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60914d-1395-47cd-8e2d-594e13ff1fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:12.505682 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a60914d-1395-47cd-8e2d-594e13ff1fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:12.522254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77714caa-c542-4475-b0c4-cbd38714ebf4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:12.534229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3a60914d-1395-47cd-8e2d-594e13ff1fb6 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:12.536808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3a60914d-1395-47cd-8e2d-594e13ff1fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:12.546572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1030/2046] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:12.551067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60914d-1395-47cd-8e2d-594e13ff1fb6 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:12.558419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997b0f3-46d4-4b17-a201-87ee0d90cbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:12.559047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997b0f3-46d4-4b17-a201-87ee0d90cbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:12.559324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997b0f3-46d4-4b17-a201-87ee0d90cbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:12.559590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997b0f3-46d4-4b17-a201-87ee0d90cbf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:12.560224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997b0f3-46d4-4b17-a201-87ee0d90cbf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:12.563889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997b0f3-46d4-4b17-a201-87ee0d90cbf3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:12.564029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a60914d-1395-47cd-8e2d-594e13ff1fb6 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:12.577523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:12.577922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:12.578272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:12.578603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:12.579410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:12.579706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:12.582334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:12.630912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1031/2047] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:12.637214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:12.649330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774db9bd-e69e-4a3c-b557-bf827bf0b204 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:12.657072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b2012-679a-482b-8255-8e8a8c196420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:12.657636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b2012-679a-482b-8255-8e8a8c196420 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:12.657913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b2012-679a-482b-8255-8e8a8c196420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:12.658230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b2012-679a-482b-8255-8e8a8c196420 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:12.664373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b2012-679a-482b-8255-8e8a8c196420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:12.668566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b2012-679a-482b-8255-8e8a8c196420 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:12.670118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1017/2048] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:12 2022] POST /identity/v3/projects => generated 367 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:12.674272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997b0f3-46d4-4b17-a201-87ee0d90cbf3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:12.811811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997b0f3-46d4-4b17-a201-87ee0d90cbf3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:12.817427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:12.818265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:12.818753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:12.819260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:12.820379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:12.822601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.100927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1032/2049] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:12 2022] POST /identity/v3/projects => generated 375 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:13.105271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b2012-679a-482b-8255-8e8a8c196420 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.171119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1018/2050] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:12 2022] POST /identity/v3/users => generated 290 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:13.172608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.174292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:13.187251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b2012-679a-482b-8255-8e8a8c196420 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.199589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4190758-3482-49e5-8499-dda8ef59afa6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.203305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57271b-fe6f-49b6-894e-c922b5acbd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.208780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.210092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.210217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57271b-fe6f-49b6-894e-c922b5acbd20 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.210695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57271b-fe6f-49b6-894e-c922b5acbd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.210855 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57271b-fe6f-49b6-894e-c922b5acbd20 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.211535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57271b-fe6f-49b6-894e-c922b5acbd20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.211897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.213283 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] PATH_INFO: `/v3/projects/40d5eede1ef14d3baefa612bf60bff16` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.213512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57271b-fe6f-49b6-894e-c922b5acbd20 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.214659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d5eede1ef14d3baefa612bf60bff16)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.217672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.396069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1033/2051] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:13 2022] POST /identity/v3/users => generated 288 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:13.396732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57271b-fe6f-49b6-894e-c922b5acbd20 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.408624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57271b-fe6f-49b6-894e-c922b5acbd20 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.438365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d5eede1ef14d3baefa612bf60bff16', 'request_id': 'req-4222e581-e382-450d-bad7-42649fd278d0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:13.530725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d5eede1ef14d3baefa612bf60bff16', 'request_id': 'req-4222e581-e382-450d-bad7-42649fd278d0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:13.538376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645cd69-c23d-4bd4-b2e2-4692f20380f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.539511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645cd69-c23d-4bd4-b2e2-4692f20380f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.539913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645cd69-c23d-4bd4-b2e2-4692f20380f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.540256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645cd69-c23d-4bd4-b2e2-4692f20380f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.541035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645cd69-c23d-4bd4-b2e2-4692f20380f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.544035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0645cd69-c23d-4bd4-b2e2-4692f20380f9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.551228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d5eede1ef14d3baefa612bf60bff16', 'request_id': 'req-4222e581-e382-450d-bad7-42649fd278d0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:13.565816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] The token cache is being invalidate because project 40d5eede1ef14d3baefa612bf60bff16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:13.566111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:13.566830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:13.570007 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1019/2052] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:13 2022] DELETE /identity/v3/projects/40d5eede1ef14d3baefa612bf60bff16 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:13.570246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1034/2053] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:13.573208 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.573645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812a7a4-2a5b-49ab-ab53-b6c3e2b270b3 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.574028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812a7a4-2a5b-49ab-ab53-b6c3e2b270b3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.574309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812a7a4-2a5b-49ab-ab53-b6c3e2b270b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.575066 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:13.590234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222e581-e382-450d-bad7-42649fd278d0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.596252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039ff57-8e3d-426b-a063-a61986fbd98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.597259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039ff57-8e3d-426b-a063-a61986fbd98d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.597685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039ff57-8e3d-426b-a063-a61986fbd98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.598028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039ff57-8e3d-426b-a063-a61986fbd98d admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.598799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039ff57-8e3d-426b-a063-a61986fbd98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.601608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a812a7a4-2a5b-49ab-ab53-b6c3e2b270b3 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:13.601939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039ff57-8e3d-426b-a063-a61986fbd98d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.603587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a812a7a4-2a5b-49ab-ab53-b6c3e2b270b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:13.613300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1035/2054] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:13.614071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812a7a4-2a5b-49ab-ab53-b6c3e2b270b3 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.615429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a812a7a4-2a5b-49ab-ab53-b6c3e2b270b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:13.620882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1020/2055] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:13.622612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7986-fcdd-4ff2-ad2a-46c475d20003 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.622899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7986-fcdd-4ff2-ad2a-46c475d20003 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.623169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba7986-fcdd-4ff2-ad2a-46c475d20003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.644712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812a7a4-2a5b-49ab-ab53-b6c3e2b270b3 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.644712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c09d79-9e3b-4163-a356-cef79be5d211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.644712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c09d79-9e3b-4163-a356-cef79be5d211 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.644712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c09d79-9e3b-4163-a356-cef79be5d211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.645094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c09d79-9e3b-4163-a356-cef79be5d211 admin admin] PATH_INFO: `/v3/projects/b64c7734dc4c4e9186cfe80fe8470dca/users/d5d0617d66e24ec89a87f16dd10ae35b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.647290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0dba7986-fcdd-4ff2-ad2a-46c475d20003 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:13.648571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0dba7986-fcdd-4ff2-ad2a-46c475d20003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:13.657862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1021/2056] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:13.658731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba7986-fcdd-4ff2-ad2a-46c475d20003 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.659053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c09d79-9e3b-4163-a356-cef79be5d211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b64c7734dc4c4e9186cfe80fe8470dca, user_id=d5d0617d66e24ec89a87f16dd10ae35b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.662153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c09d79-9e3b-4163-a356-cef79be5d211 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.668177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba7986-fcdd-4ff2-ad2a-46c475d20003 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.673488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d8906-d498-4a06-a980-6528ea5d4c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.674071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d8906-d498-4a06-a980-6528ea5d4c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.674361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d8906-d498-4a06-a980-6528ea5d4c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.674635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d8906-d498-4a06-a980-6528ea5d4c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.675261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d8906-d498-4a06-a980-6528ea5d4c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.676859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d8906-d498-4a06-a980-6528ea5d4c27 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.686813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1022/2057] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:13.687937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d8906-d498-4a06-a980-6528ea5d4c27 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.697636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d8906-d498-4a06-a980-6528ea5d4c27 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.702366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.702962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.703903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.704175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.704797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:13.705091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.706697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.718601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1023/2058] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:13.720590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.733179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cda00-c93c-4e4e-8a08-3a27a5a047b7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.744816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1036/2059] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:13 2022] PUT /identity/v3/projects/b64c7734dc4c4e9186cfe80fe8470dca/users/d5d0617d66e24ec89a87f16dd10ae35b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:13.745678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c09d79-9e3b-4163-a356-cef79be5d211 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:13.822793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb005b-5920-4171-b6d3-0c8e67115f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.823613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb005b-5920-4171-b6d3-0c8e67115f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.823900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb005b-5920-4171-b6d3-0c8e67115f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.824175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb005b-5920-4171-b6d3-0c8e67115f76 admin admin] PATH_INFO: `/v3/projects/000036f1c29a429eb76853dadde07e4c/users/fff7c8c5a12f43219f023656c0085714/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.864005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c09d79-9e3b-4163-a356-cef79be5d211 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:13.874102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb005b-5920-4171-b6d3-0c8e67115f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=000036f1c29a429eb76853dadde07e4c, user_id=fff7c8c5a12f43219f023656c0085714, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.877409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb005b-5920-4171-b6d3-0c8e67115f76 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.887271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f53ff4-a54f-41c7-bb12-7ae265ff4da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:13.891217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f53ff4-a54f-41c7-bb12-7ae265ff4da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:13.891217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f53ff4-a54f-41c7-bb12-7ae265ff4da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:13.891217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f53ff4-a54f-41c7-bb12-7ae265ff4da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:13.891217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f53ff4-a54f-41c7-bb12-7ae265ff4da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:13.891997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f53ff4-a54f-41c7-bb12-7ae265ff4da4 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:13.993563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1024/2060] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:13 2022] PUT /identity/v3/projects/000036f1c29a429eb76853dadde07e4c/users/fff7c8c5a12f43219f023656c0085714/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:13.994462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb005b-5920-4171-b6d3-0c8e67115f76 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.003116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1037/2061] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:13 2022] POST /identity/v3/projects => generated 383 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:14.004350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798786-bf03-46ea-8488-2480ed312bfa None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.014300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798786-bf03-46ea-8488-2480ed312bfa None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.015118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798786-bf03-46ea-8488-2480ed312bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.050170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-77798786-bf03-46ea-8488-2480ed312bfa None None] MFA Rules not processed for user `fff7c8c5a12f43219f023656c0085714`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:14.051443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-77798786-bf03-46ea-8488-2480ed312bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:14.069525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb005b-5920-4171-b6d3-0c8e67115f76 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.120697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1038/2062] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:14.121565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77798786-bf03-46ea-8488-2480ed312bfa None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.138818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef02baa-5c7e-42f8-91cb-af59af8f33e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.146036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef02baa-5c7e-42f8-91cb-af59af8f33e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.146036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef02baa-5c7e-42f8-91cb-af59af8f33e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.146036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef02baa-5c7e-42f8-91cb-af59af8f33e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.146036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef02baa-5c7e-42f8-91cb-af59af8f33e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.146036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef02baa-5c7e-42f8-91cb-af59af8f33e7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.155585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1025/2063] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:14.157415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef02baa-5c7e-42f8-91cb-af59af8f33e7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.172731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef02baa-5c7e-42f8-91cb-af59af8f33e7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.177509 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113041b-1325-43b6-b05f-d4ed9cb16119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.180750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113041b-1325-43b6-b05f-d4ed9cb16119 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.180750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113041b-1325-43b6-b05f-d4ed9cb16119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.180750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113041b-1325-43b6-b05f-d4ed9cb16119 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.180750 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113041b-1325-43b6-b05f-d4ed9cb16119 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.181254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b113041b-1325-43b6-b05f-d4ed9cb16119 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.196545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77798786-bf03-46ea-8488-2480ed312bfa None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.201494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ead06b-cc19-4e40-be82-f373a267bc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.202138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ead06b-cc19-4e40-be82-f373a267bc83 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.202432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ead06b-cc19-4e40-be82-f373a267bc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.209427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ead06b-cc19-4e40-be82-f373a267bc83 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.209427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ead06b-cc19-4e40-be82-f373a267bc83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.209427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ead06b-cc19-4e40-be82-f373a267bc83 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.255612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:14.293439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1026/2064] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:14 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:14.294259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113041b-1325-43b6-b05f-d4ed9cb16119 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.356881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1039/2065] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:14 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:14.357616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ead06b-cc19-4e40-be82-f373a267bc83 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.364217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b113041b-1325-43b6-b05f-d4ed9cb16119 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.367279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ead06b-cc19-4e40-be82-f373a267bc83 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.369140 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26eb872-ada3-4a7a-addc-c5703da14fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.369911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eb872-ada3-4a7a-addc-c5703da14fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.370181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eb872-ada3-4a7a-addc-c5703da14fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.370484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26eb872-ada3-4a7a-addc-c5703da14fa0 admin admin] PATH_INFO: `/v3/projects/b64c7734dc4c4e9186cfe80fe8470dca/users/d5d0617d66e24ec89a87f16dd10ae35b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.371880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de480b-daf3-4ebd-a4ba-eccb91cbad18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.372366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eb872-ada3-4a7a-addc-c5703da14fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b64c7734dc4c4e9186cfe80fe8470dca, user_id=d5d0617d66e24ec89a87f16dd10ae35b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.372466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de480b-daf3-4ebd-a4ba-eccb91cbad18 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.372718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de480b-daf3-4ebd-a4ba-eccb91cbad18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.372988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de480b-daf3-4ebd-a4ba-eccb91cbad18 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.373603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de480b-daf3-4ebd-a4ba-eccb91cbad18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.375183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26eb872-ada3-4a7a-addc-c5703da14fa0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.375799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de480b-daf3-4ebd-a4ba-eccb91cbad18 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.386937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1027/2066] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:14 2022] PUT /identity/v3/projects/b64c7734dc4c4e9186cfe80fe8470dca/users/d5d0617d66e24ec89a87f16dd10ae35b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:14.391220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26eb872-ada3-4a7a-addc-c5703da14fa0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.449912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26eb872-ada3-4a7a-addc-c5703da14fa0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.455385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630ee07-6f7e-4075-a8c0-ce81677e5821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.455988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630ee07-6f7e-4075-a8c0-ce81677e5821 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.456328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630ee07-6f7e-4075-a8c0-ce81677e5821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.456703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630ee07-6f7e-4075-a8c0-ce81677e5821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.457343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630ee07-6f7e-4075-a8c0-ce81677e5821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.458982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630ee07-6f7e-4075-a8c0-ce81677e5821 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.465687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1040/2067] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:14 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:14.472861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1028/2068] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:14.473141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16591-d540-47d4-ab65-ece037e77dd3 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.473141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16591-d540-47d4-ab65-ece037e77dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.473141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e16591-d540-47d4-ab65-ece037e77dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.473361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630ee07-6f7e-4075-a8c0-ce81677e5821 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.480930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630ee07-6f7e-4075-a8c0-ce81677e5821 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.487018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec684130-23f4-4101-a224-2cdf1e8068f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.487747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec684130-23f4-4101-a224-2cdf1e8068f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.488074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec684130-23f4-4101-a224-2cdf1e8068f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.488348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec684130-23f4-4101-a224-2cdf1e8068f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.488970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec684130-23f4-4101-a224-2cdf1e8068f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.490598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec684130-23f4-4101-a224-2cdf1e8068f7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.500994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-b2e16591-d540-47d4-ab65-ece037e77dd3 None None] MFA Rules not processed for user `d5d0617d66e24ec89a87f16dd10ae35b`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:14.505121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1029/2069] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:14.505280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b2e16591-d540-47d4-ab65-ece037e77dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:14.505421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec684130-23f4-4101-a224-2cdf1e8068f7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.512383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec684130-23f4-4101-a224-2cdf1e8068f7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.518149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334aef2-8735-4ffc-9f47-121ea9cecb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.519067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334aef2-8735-4ffc-9f47-121ea9cecb10 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.519442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334aef2-8735-4ffc-9f47-121ea9cecb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.519813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334aef2-8735-4ffc-9f47-121ea9cecb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.520651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334aef2-8735-4ffc-9f47-121ea9cecb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.523292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334aef2-8735-4ffc-9f47-121ea9cecb10 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.536291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1030/2070] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:14.539307 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334aef2-8735-4ffc-9f47-121ea9cecb10 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.550172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334aef2-8735-4ffc-9f47-121ea9cecb10 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.555069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.555714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.555990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.556273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] PATH_INFO: `/v3/projects/1b24364202a44dd5a936fca370419e28/users/a287d19ad411461aa7025bed1c695175/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.571505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b24364202a44dd5a936fca370419e28, user_id=a287d19ad411461aa7025bed1c695175, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.574174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.575532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1041/2071] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:14.576650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16591-d540-47d4-ab65-ece037e77dd3 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.587650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e16591-d540-47d4-ab65-ece037e77dd3 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.592357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb848a6-2869-4a19-9a23-a2aa7fa54f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.594958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb848a6-2869-4a19-9a23-a2aa7fa54f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.594958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb848a6-2869-4a19-9a23-a2aa7fa54f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.594958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb848a6-2869-4a19-9a23-a2aa7fa54f85 admin admin] PATH_INFO: `/v3/projects/5849943721724643b980baffe607c43d/users/ab8a0387e8404000937eb6a02b1aed0c/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.609372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb848a6-2869-4a19-9a23-a2aa7fa54f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5849943721724643b980baffe607c43d, user_id=ab8a0387e8404000937eb6a02b1aed0c, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.612129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb848a6-2869-4a19-9a23-a2aa7fa54f85 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.676096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1031/2072] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:14 2022] PUT /identity/v3/projects/1b24364202a44dd5a936fca370419e28/users/a287d19ad411461aa7025bed1c695175/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:14.679896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.679896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:14.737859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1042/2073] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:14 2022] PUT /identity/v3/projects/5849943721724643b980baffe607c43d/users/ab8a0387e8404000937eb6a02b1aed0c/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:14.739191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb848a6-2869-4a19-9a23-a2aa7fa54f85 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.757504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa502c-2f55-4275-b488-38997eac6553 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.811359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb848a6-2869-4a19-9a23-a2aa7fa54f85 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.814762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110a546-d73b-4e06-9be0-5d6f9a05bca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.815699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110a546-d73b-4e06-9be0-5d6f9a05bca4 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.815970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110a546-d73b-4e06-9be0-5d6f9a05bca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.816233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110a546-d73b-4e06-9be0-5d6f9a05bca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.826012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2332098-7d27-4f20-8f85-7363bf1621eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.826864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2332098-7d27-4f20-8f85-7363bf1621eb admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.827142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2332098-7d27-4f20-8f85-7363bf1621eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.827626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2332098-7d27-4f20-8f85-7363bf1621eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.829024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2332098-7d27-4f20-8f85-7363bf1621eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.831776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2332098-7d27-4f20-8f85-7363bf1621eb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.842385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1043/2074] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:14.847368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a2ce4-ec20-4ceb-af2b-eb8a20f43c2a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.847368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a2ce4-ec20-4ceb-af2b-eb8a20f43c2a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.847368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a2ce4-ec20-4ceb-af2b-eb8a20f43c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.847649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:14.881717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-db2a2ce4-ec20-4ceb-af2b-eb8a20f43c2a None None] MFA Rules not processed for user `ab8a0387e8404000937eb6a02b1aed0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:14.881717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-db2a2ce4-ec20-4ceb-af2b-eb8a20f43c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:14.907335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110a546-d73b-4e06-9be0-5d6f9a05bca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.909069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110a546-d73b-4e06-9be0-5d6f9a05bca4 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.920198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1032/2075] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:14.921207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110a546-d73b-4e06-9be0-5d6f9a05bca4 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:14.947960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110a546-d73b-4e06-9be0-5d6f9a05bca4 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:14.955462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f3c24-4624-4c29-940b-e79436e28bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:14.959931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3c24-4624-4c29-940b-e79436e28bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:14.959931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3c24-4624-4c29-940b-e79436e28bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:14.959931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f3c24-4624-4c29-940b-e79436e28bc0 admin admin] PATH_INFO: `/v3/projects/1b24364202a44dd5a936fca370419e28/users/a287d19ad411461aa7025bed1c695175/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:14.959931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f3c24-4624-4c29-940b-e79436e28bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b24364202a44dd5a936fca370419e28, user_id=a287d19ad411461aa7025bed1c695175, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:14.961664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f3c24-4624-4c29-940b-e79436e28bc0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:14.980587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1033/2076] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:14 2022] PUT /identity/v3/projects/1b24364202a44dd5a936fca370419e28/users/a287d19ad411461aa7025bed1c695175/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:14.991016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1044/2077] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:15.002694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871b2c2-42ba-4775-be1f-8024eb805919 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.003047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871b2c2-42ba-4775-be1f-8024eb805919 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.003378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871b2c2-42ba-4775-be1f-8024eb805919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.033692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2871b2c2-42ba-4775-be1f-8024eb805919 None None] MFA Rules not processed for user `a287d19ad411461aa7025bed1c695175`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:15.035242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2871b2c2-42ba-4775-be1f-8024eb805919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.055693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a2ce4-ec20-4ceb-af2b-eb8a20f43c2a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:15.057245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-db2a2ce4-ec20-4ceb-af2b-eb8a20f43c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.073455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a2ce4-ec20-4ceb-af2b-eb8a20f43c2a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:15.142031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1034/2078] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:15.147448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871b2c2-42ba-4775-be1f-8024eb805919 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:15.148760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:15.149431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.149718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.149988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] PATH_INFO: `/v3/users/3ee0edb0233b4a499191efe0b2f9561f` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.151393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ee0edb0233b4a499191efe0b2f9561f)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:15.151535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2871b2c2-42ba-4775-be1f-8024eb805919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.153294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:15.217391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871b2c2-42ba-4775-be1f-8024eb805919 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:15.221591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351be6e-55c3-4892-bd35-0a313b1048ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:15.222188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351be6e-55c3-4892-bd35-0a313b1048ca service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.222503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351be6e-55c3-4892-bd35-0a313b1048ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.222767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351be6e-55c3-4892-bd35-0a313b1048ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.277349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351be6e-55c3-4892-bd35-0a313b1048ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:15.278896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351be6e-55c3-4892-bd35-0a313b1048ca service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:15.288855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1035/2079] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:15.292954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351be6e-55c3-4892-bd35-0a313b1048ca service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:15.298843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351be6e-55c3-4892-bd35-0a313b1048ca service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:15.307033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:15.307658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.307939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.308211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.310716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.371091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:15.372531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:15.377591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ee0edb0233b4a499191efe0b2f9561f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.378236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ee0edb0233b4a499191efe0b2f9561f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.381894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1036/2080] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:15.382794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:15.393597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c0afc-9e44-4cf7-bc3f-c8614d05037f service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:15.398201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:15.398851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.399125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.399396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] PATH_INFO: `/v3/users/ab8a0387e8404000937eb6a02b1aed0c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.400545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab8a0387e8404000937eb6a02b1aed0c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:15.402636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:15.463609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ee0edb0233b4a499191efe0b2f9561f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.477663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ee0edb0233b4a499191efe0b2f9561f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.483980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:15.487630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ee0edb0233b4a499191efe0b2f9561f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.494380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1045/2081] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:15 2022] DELETE /identity/v3/users/3ee0edb0233b4a499191efe0b2f9561f => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:15.498119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:15.502879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.558396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab8a0387e8404000937eb6a02b1aed0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.559768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a372a-04c1-4532-b61f-293be6bc6344 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:15.563707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9193c23-40e1-4a76-bad2-c15a89fd364b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:15.564318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9193c23-40e1-4a76-bad2-c15a89fd364b service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.564596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9193c23-40e1-4a76-bad2-c15a89fd364b service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.564867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9193c23-40e1-4a76-bad2-c15a89fd364b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.566768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b9193c23-40e1-4a76-bad2-c15a89fd364b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.626869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9193c23-40e1-4a76-bad2-c15a89fd364b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:15.628296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9193c23-40e1-4a76-bad2-c15a89fd364b service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:15.632225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab8a0387e8404000937eb6a02b1aed0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.632911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab8a0387e8404000937eb6a02b1aed0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.637368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1046/2082] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:15.638779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07095-7c07-4c9a-af88-b5c52d45e450 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.647092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07095-7c07-4c9a-af88-b5c52d45e450 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.647092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b07095-7c07-4c9a-af88-b5c52d45e450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.647665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab8a0387e8404000937eb6a02b1aed0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.656807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-989e8e83-d466-44cf-89dc-53fe5fb684d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab8a0387e8404000937eb6a02b1aed0c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:15.660567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d4b07095-7c07-4c9a-af88-b5c52d45e450 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:15.661773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d4b07095-7c07-4c9a-af88-b5c52d45e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.663444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1037/2083] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:15 2022] DELETE /identity/v3/users/ab8a0387e8404000937eb6a02b1aed0c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:15.667905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa16ec-0396-457d-9b99-a97b74c93ff1 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.668281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa16ec-0396-457d-9b99-a97b74c93ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.668686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa16ec-0396-457d-9b99-a97b74c93ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.692964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1dfa16ec-0396-457d-9b99-a97b74c93ff1 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:15.694435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1dfa16ec-0396-457d-9b99-a97b74c93ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.748350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1047/2084] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:15.758716 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b07095-7c07-4c9a-af88-b5c52d45e450 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:15.760090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d4b07095-7c07-4c9a-af88-b5c52d45e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.770586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1038/2085] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:15.780933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa16ec-0396-457d-9b99-a97b74c93ff1 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:15.783938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1dfa16ec-0396-457d-9b99-a97b74c93ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:15.826221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b07095-7c07-4c9a-af88-b5c52d45e450 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:15.830764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:15.831383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.831652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.831919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.842329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:15.845986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:15.858440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:15.871378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1048/2086] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:15.884503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa16ec-0396-457d-9b99-a97b74c93ff1 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:15.891137 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:15.891789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:15.892067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:15.892335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:15.901835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:15.903309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:15.914578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1039/2087] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:16.573500 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:16.575215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:16.590499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5f1b8-052b-4c55-9b7b-a1a306617589 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:16.595769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:16.596454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:16.596952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:16.597065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] PATH_INFO: `/v3/projects/172c9af47b7e4294860031d5747c1a66` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:16.598377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=172c9af47b7e4294860031d5747c1a66)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:16.600134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:16.623786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:16.629751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:16.651122 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed406e-1961-42fd-8968-784b33227f92 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:16.658944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:16.658944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:16.658944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:16.658944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] PATH_INFO: `/v3/projects/5849943721724643b980baffe607c43d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:16.659587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5849943721724643b980baffe607c43d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:16.661519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:16.805666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172c9af47b7e4294860031d5747c1a66', 'request_id': 'req-bc09fa81-348c-434a-b5d5-3e66a44375c7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:16.852046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5849943721724643b980baffe607c43d', 'request_id': 'req-7679df37-3669-470a-a7c2-167c24d6ff6a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:16.903614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172c9af47b7e4294860031d5747c1a66', 'request_id': 'req-bc09fa81-348c-434a-b5d5-3e66a44375c7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:16.911016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172c9af47b7e4294860031d5747c1a66', 'request_id': 'req-bc09fa81-348c-434a-b5d5-3e66a44375c7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:16.923806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] The token cache is being invalidate because project 172c9af47b7e4294860031d5747c1a66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:16.924096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:16.924829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:16.926591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1049/2088] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:16 2022] DELETE /identity/v3/projects/172c9af47b7e4294860031d5747c1a66 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:16.930236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:16.931679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:16.989489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5849943721724643b980baffe607c43d', 'request_id': 'req-7679df37-3669-470a-a7c2-167c24d6ff6a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:16.996198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5849943721724643b980baffe607c43d', 'request_id': 'req-7679df37-3669-470a-a7c2-167c24d6ff6a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:16.998875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09fa81-348c-434a-b5d5-3e66a44375c7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:17.009237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] The token cache is being invalidate because project 5849943721724643b980baffe607c43d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:17.009527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:17.010202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7679df37-3669-470a-a7c2-167c24d6ff6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:17.011764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1040/2089] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:16 2022] DELETE /identity/v3/projects/5849943721724643b980baffe607c43d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:17.017033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09117632-f3f3-439b-af4b-66731451d32e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.017322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09117632-f3f3-439b-af4b-66731451d32e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.017596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09117632-f3f3-439b-af4b-66731451d32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.037555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-09117632-f3f3-439b-af4b-66731451d32e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:17.038787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-09117632-f3f3-439b-af4b-66731451d32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:17.055559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:17.056212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.056477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.056744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] PATH_INFO: `/v3/users/2f54fffeb9604ecd945490263c9aec9b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.057890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f54fffeb9604ecd945490263c9aec9b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:17.059879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:17.115153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1041/2090] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:17.123758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b86cd-01d3-4baf-b59c-f392b35a8566 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.124681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b86cd-01d3-4baf-b59c-f392b35a8566 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.124681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b86cd-01d3-4baf-b59c-f392b35a8566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.145222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-291b86cd-01d3-4baf-b59c-f392b35a8566 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:17.146449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-291b86cd-01d3-4baf-b59c-f392b35a8566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:17.157973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1042/2091] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:17.159096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b86cd-01d3-4baf-b59c-f392b35a8566 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:17.167784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b86cd-01d3-4baf-b59c-f392b35a8566 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:17.172677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:17.173255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.173522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.173789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.174446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:17.174740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:17.176354 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:17.191930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1043/2092] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:22:17.192529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:17.202190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdcc2e-b695-4516-a6fa-5ad85d726c5b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:17.208049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc40ef9-a80d-4d0a-b581-daf4d8cf9941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:17.208645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc40ef9-a80d-4d0a-b581-daf4d8cf9941 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.208915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc40ef9-a80d-4d0a-b581-daf4d8cf9941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.209177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc40ef9-a80d-4d0a-b581-daf4d8cf9941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.209804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc40ef9-a80d-4d0a-b581-daf4d8cf9941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:17.211710 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc40ef9-a80d-4d0a-b581-daf4d8cf9941 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:17.320609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1044/2093] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:17 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:17.324649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc40ef9-a80d-4d0a-b581-daf4d8cf9941 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:17.357574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f54fffeb9604ecd945490263c9aec9b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:17.358672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f54fffeb9604ecd945490263c9aec9b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:17.426468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc40ef9-a80d-4d0a-b581-daf4d8cf9941 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:17.433848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb09b4a-e008-495b-9b58-48d4d7d7b27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:17.434579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb09b4a-e008-495b-9b58-48d4d7d7b27c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.434948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb09b4a-e008-495b-9b58-48d4d7d7b27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.435291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb09b4a-e008-495b-9b58-48d4d7d7b27c admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.435996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb09b4a-e008-495b-9b58-48d4d7d7b27c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:17.438927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb09b4a-e008-495b-9b58-48d4d7d7b27c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:17.476024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f54fffeb9604ecd945490263c9aec9b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:17.492538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f54fffeb9604ecd945490263c9aec9b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:17.506059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f54fffeb9604ecd945490263c9aec9b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:17.514424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1050/2094] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:16 2022] DELETE /identity/v3/users/2f54fffeb9604ecd945490263c9aec9b => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:17.557966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1045/2095] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:17 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:17.563176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:17.565144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:17.580684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:17.704910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa317839-4631-4a5a-ac03-52bc9d13472f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:17.708769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13ab74-a253-472a-ab0b-f3b0059ce001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:17.709533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13ab74-a253-472a-ab0b-f3b0059ce001 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.709813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13ab74-a253-472a-ab0b-f3b0059ce001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.710090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13ab74-a253-472a-ab0b-f3b0059ce001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.712668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13ab74-a253-472a-ab0b-f3b0059ce001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:17.714950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13ab74-a253-472a-ab0b-f3b0059ce001 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:17.733882 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1051/2096] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:17.734019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb09b4a-e008-495b-9b58-48d4d7d7b27c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:17.752154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb09b4a-e008-495b-9b58-48d4d7d7b27c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:17.757558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c352b13-d734-4c7b-87bb-b9c1f52fd976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:17.758174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352b13-d734-4c7b-87bb-b9c1f52fd976 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.758641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352b13-d734-4c7b-87bb-b9c1f52fd976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.758772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c352b13-d734-4c7b-87bb-b9c1f52fd976 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.759411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c352b13-d734-4c7b-87bb-b9c1f52fd976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:17.761022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c352b13-d734-4c7b-87bb-b9c1f52fd976 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:17.772605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1046/2097] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:22:17.776844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13ab74-a253-472a-ab0b-f3b0059ce001 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:17.788606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13ab74-a253-472a-ab0b-f3b0059ce001 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:17.800012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:17.805571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.805571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.805571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] PATH_INFO: `/v3/projects/de3172ce471840bba0241dbebfda2a8a/users/697f296a14b44f8cbc5a3ff3c0fc18f1/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.828341 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3172ce471840bba0241dbebfda2a8a, user_id=697f296a14b44f8cbc5a3ff3c0fc18f1, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:17.833106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:17.916103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1052/2098] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:17 2022] PUT /identity/v3/projects/de3172ce471840bba0241dbebfda2a8a/users/697f296a14b44f8cbc5a3ff3c0fc18f1/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:17.925829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec013e0-e37b-49cc-b8b5-dbf4cac0a87a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:17.925942 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec013e0-e37b-49cc-b8b5-dbf4cac0a87a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:17.926210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec013e0-e37b-49cc-b8b5-dbf4cac0a87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:17.973503 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8ec013e0-e37b-49cc-b8b5-dbf4cac0a87a None None] MFA Rules not processed for user `697f296a14b44f8cbc5a3ff3c0fc18f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:17.974867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8ec013e0-e37b-49cc-b8b5-dbf4cac0a87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:18.045316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1047/2099] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:18.057288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:18.058654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:18.067360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec013e0-e37b-49cc-b8b5-dbf4cac0a87a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:18.070856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8ec013e0-e37b-49cc-b8b5-dbf4cac0a87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:18.087521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:18.145799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ff712-4a6e-4579-a28a-802a5e5c715e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:18.149544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c9ff-4b46-4ab1-b658-81047d991a9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:18.150155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c9ff-4b46-4ab1-b658-81047d991a9a service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:18.150466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c9ff-4b46-4ab1-b658-81047d991a9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:18.151026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c9ff-4b46-4ab1-b658-81047d991a9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:18.158958 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec013e0-e37b-49cc-b8b5-dbf4cac0a87a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:18.161798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0c9ff-4b46-4ab1-b658-81047d991a9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:18.163370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0c9ff-4b46-4ab1-b658-81047d991a9a service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:18.163881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:18.164589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:18.164934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:18.165286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] PATH_INFO: `/v3/projects/0718e11b149b4f8093ede261f62e4225` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:18.166584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0718e11b149b4f8093ede261f62e4225)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:18.168780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:18.173118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1053/2100] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:18.297827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0718e11b149b4f8093ede261f62e4225', 'request_id': 'req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.389980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0718e11b149b4f8093ede261f62e4225', 'request_id': 'req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.398732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0718e11b149b4f8093ede261f62e4225', 'request_id': 'req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.413492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] The token cache is being invalidate because project 0718e11b149b4f8093ede261f62e4225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:18.413824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.414491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.416265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1048/2101] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:18 2022] DELETE /identity/v3/projects/0718e11b149b4f8093ede261f62e4225 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:18.420563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c9ff-4b46-4ab1-b658-81047d991a9a service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:18.438720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c9ff-4b46-4ab1-b658-81047d991a9a service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:18.520925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:18.521561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:18.521837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:18.522187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] PATH_INFO: `/v3/users/6991b5602fcf43908a549d0c0ae64ab5` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:18.523387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6991b5602fcf43908a549d0c0ae64ab5)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:18.525358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:18.736632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6991b5602fcf43908a549d0c0ae64ab5'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.737542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6991b5602fcf43908a549d0c0ae64ab5'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.803970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6991b5602fcf43908a549d0c0ae64ab5'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.817702 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6991b5602fcf43908a549d0c0ae64ab5'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.824645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6991b5602fcf43908a549d0c0ae64ab5'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:18.831768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1054/2102] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:18 2022] DELETE /identity/v3/users/6991b5602fcf43908a549d0c0ae64ab5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:18.990374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:18.991864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:19.011725 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:19.016744 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf5e7d-a34c-4e37-a388-4eb20bf6678a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:19.105450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:19.110757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:19.110757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:19.110757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] PATH_INFO: `/v3/users/fff7c8c5a12f43219f023656c0085714` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:19.110757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff7c8c5a12f43219f023656c0085714)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:19.110757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:19.397917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff7c8c5a12f43219f023656c0085714'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.426379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:19.439382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:19.455861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad75cc-d9a2-45c7-bf36-188774b3450b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:19.462943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:19.465233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:19.465724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:19.466193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] PATH_INFO: `/v3/projects/a9011a3b50f34e1da995b59d9053306b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:19.467605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9011a3b50f34e1da995b59d9053306b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:19.470445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:19.473225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff7c8c5a12f43219f023656c0085714'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.473982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff7c8c5a12f43219f023656c0085714'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.482612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff7c8c5a12f43219f023656c0085714'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.496064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2cac9a12-8a36-4556-85a6-5157947f23aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff7c8c5a12f43219f023656c0085714'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.503890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1049/2103] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:18 2022] DELETE /identity/v3/users/fff7c8c5a12f43219f023656c0085714 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:19.508284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0d7c-5a0e-4a2d-9f40-773f7e6fa083 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:19.508569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0d7c-5a0e-4a2d-9f40-773f7e6fa083 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:19.508840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0d7c-5a0e-4a2d-9f40-773f7e6fa083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:19.529202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b9cb0d7c-5a0e-4a2d-9f40-773f7e6fa083 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:19.533219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b9cb0d7c-5a0e-4a2d-9f40-773f7e6fa083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:19.540292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:19.602596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1050/2104] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:19.629843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb0d7c-5a0e-4a2d-9f40-773f7e6fa083 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:19.631492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b9cb0d7c-5a0e-4a2d-9f40-773f7e6fa083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:19.660363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9011a3b50f34e1da995b59d9053306b', 'request_id': 'req-169e1c68-f253-48a8-8e82-7c4896614671'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.738212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb0d7c-5a0e-4a2d-9f40-773f7e6fa083 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:19.740628 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9011a3b50f34e1da995b59d9053306b', 'request_id': 'req-169e1c68-f253-48a8-8e82-7c4896614671'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.756942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9011a3b50f34e1da995b59d9053306b', 'request_id': 'req-169e1c68-f253-48a8-8e82-7c4896614671'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.780302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] The token cache is being invalidate because project a9011a3b50f34e1da995b59d9053306b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:19.780596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.781317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-169e1c68-f253-48a8-8e82-7c4896614671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:19.783145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1055/2105] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:19 2022] DELETE /identity/v3/projects/a9011a3b50f34e1da995b59d9053306b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:19.788737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ea4d2-a8a5-476a-bf25-af925d87b904 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:19.789023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ea4d2-a8a5-476a-bf25-af925d87b904 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:19.789297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ea4d2-a8a5-476a-bf25-af925d87b904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:19.831934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-509ea4d2-a8a5-476a-bf25-af925d87b904 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:19.833329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-509ea4d2-a8a5-476a-bf25-af925d87b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:19.861349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:19.862226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:19.862470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:19.862691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:19.866935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:19.933753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1056/2106] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:19.944712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aef1bc-e33f-4aa8-860c-c2752189429b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:19.950122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aef1bc-e33f-4aa8-860c-c2752189429b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:19.950745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aef1bc-e33f-4aa8-860c-c2752189429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:19.999143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-56aef1bc-e33f-4aa8-860c-c2752189429b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:20.000441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-56aef1bc-e33f-4aa8-860c-c2752189429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:20.012164 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1057/2107] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:20.014169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aef1bc-e33f-4aa8-860c-c2752189429b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:20.062215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:20.064916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:20.080909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aef1bc-e33f-4aa8-860c-c2752189429b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:20.086998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:20.087886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:20.088414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:20.088849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:20.089600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1051/2108] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:20.089865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:20.090472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:20.092302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:20.111581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1058/2109] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:20.113691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:20.124383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1d735-3906-44e9-be84-c55a5551d0ef service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:20.133764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4e0a5-9b07-40e8-b7ef-f39fa02202bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:20.134763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4e0a5-9b07-40e8-b7ef-f39fa02202bd admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:20.135251 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4e0a5-9b07-40e8-b7ef-f39fa02202bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:20.135579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4e0a5-9b07-40e8-b7ef-f39fa02202bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:20.136593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4e0a5-9b07-40e8-b7ef-f39fa02202bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:20.139672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4e0a5-9b07-40e8-b7ef-f39fa02202bd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:20.255352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:20.262495 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1052/2110] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:20 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:20.267657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:20.279454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20973e-e3ab-440d-a5ef-e11edd4492c8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:20.413459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690374f9-9691-4420-a8df-7f4334993b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:20.415415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690374f9-9691-4420-a8df-7f4334993b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:20.415865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690374f9-9691-4420-a8df-7f4334993b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:20.416468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690374f9-9691-4420-a8df-7f4334993b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:20.420360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690374f9-9691-4420-a8df-7f4334993b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:20.427109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690374f9-9691-4420-a8df-7f4334993b63 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:20.541319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1059/2111] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:20 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:20.549233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4e0a5-9b07-40e8-b7ef-f39fa02202bd admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:20.561691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4e0a5-9b07-40e8-b7ef-f39fa02202bd admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:20.569041 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:20.569674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:20.571453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:20.571746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:20.572492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:20.574128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:20.588376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1053/2112] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:20.593313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690374f9-9691-4420-a8df-7f4334993b63 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:20.609539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690374f9-9691-4420-a8df-7f4334993b63 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:20.645930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:20.647908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:20.663947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a45fbd-0633-46d6-bca5-c0f0d1e537b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:20.670549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535c562-425b-4e73-8e30-b483803d6897 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:20.672336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45fbd-0633-46d6-bca5-c0f0d1e537b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:20.675127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45fbd-0633-46d6-bca5-c0f0d1e537b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:20.677379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a45fbd-0633-46d6-bca5-c0f0d1e537b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:20.680295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45fbd-0633-46d6-bca5-c0f0d1e537b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:20.683230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:20.683912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:20.688347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a45fbd-0633-46d6-bca5-c0f0d1e537b6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:20.688532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:20.688532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] PATH_INFO: `/v3/projects/000036f1c29a429eb76853dadde07e4c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:20.689613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=000036f1c29a429eb76853dadde07e4c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:20.692591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:20.710159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:20.712693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1060/2113] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:20.713745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a45fbd-0633-46d6-bca5-c0f0d1e537b6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:20.745805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a45fbd-0633-46d6-bca5-c0f0d1e537b6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:20.769590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17526a-f3ff-4bd4-abdf-4051e3f1f024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:20.770599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17526a-f3ff-4bd4-abdf-4051e3f1f024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:20.770973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17526a-f3ff-4bd4-abdf-4051e3f1f024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:20.771343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17526a-f3ff-4bd4-abdf-4051e3f1f024 admin admin] PATH_INFO: `/v3/projects/75569f93065d4dca9fc67911dd786792/users/9f801d273c7c44ecaa3cb05fcf982d0a/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:20.795679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17526a-f3ff-4bd4-abdf-4051e3f1f024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75569f93065d4dca9fc67911dd786792, user_id=9f801d273c7c44ecaa3cb05fcf982d0a, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:20.799674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17526a-f3ff-4bd4-abdf-4051e3f1f024 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:20.931476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1061/2114] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:20 2022] PUT /identity/v3/projects/75569f93065d4dca9fc67911dd786792/users/9f801d273c7c44ecaa3cb05fcf982d0a/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:20.934663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '000036f1c29a429eb76853dadde07e4c', 'request_id': 'req-1153f163-3523-4a79-9da8-3c5d0010fc4f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:20.943719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692aa34-4568-48e9-8c22-f89f202121e4 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:20.944084 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692aa34-4568-48e9-8c22-f89f202121e4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:20.944427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692aa34-4568-48e9-8c22-f89f202121e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:20.972816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-9692aa34-4568-48e9-8c22-f89f202121e4 None None] MFA Rules not processed for user `9f801d273c7c44ecaa3cb05fcf982d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:20.974093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9692aa34-4568-48e9-8c22-f89f202121e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:21.004176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '000036f1c29a429eb76853dadde07e4c', 'request_id': 'req-1153f163-3523-4a79-9da8-3c5d0010fc4f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.013611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '000036f1c29a429eb76853dadde07e4c', 'request_id': 'req-1153f163-3523-4a79-9da8-3c5d0010fc4f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.032213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] The token cache is being invalidate because project 000036f1c29a429eb76853dadde07e4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:21.032976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.033976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.035986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1054/2115] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:20 2022] DELETE /identity/v3/projects/000036f1c29a429eb76853dadde07e4c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:21.039923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:21.041582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:21.063747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1062/2116] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:21.119353 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153f163-3523-4a79-9da8-3c5d0010fc4f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:21.198576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692aa34-4568-48e9-8c22-f89f202121e4 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:21.200388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9692aa34-4568-48e9-8c22-f89f202121e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:21.204138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:21.205031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:21.205469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:21.205930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] PATH_INFO: `/v3/users/a287d19ad411461aa7025bed1c695175` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:21.207721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a287d19ad411461aa7025bed1c695175)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:21.211811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:21.304821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692aa34-4568-48e9-8c22-f89f202121e4 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:21.309365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:21.311013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:21.311613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:21.314493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:21.330060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:21.331705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:21.345189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1063/2117] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:21.503423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a287d19ad411461aa7025bed1c695175'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.594713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a287d19ad411461aa7025bed1c695175'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.595304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a287d19ad411461aa7025bed1c695175'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.603023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a287d19ad411461aa7025bed1c695175'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.617633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b4dcedaa-9c4e-402a-b750-e67cfcad7f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a287d19ad411461aa7025bed1c695175'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:21.624265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:21.629514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1055/2118] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:21 2022] DELETE /identity/v3/users/a287d19ad411461aa7025bed1c695175 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:21.699402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:21.701106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:21.722775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3882b5-03f9-4329-aead-fda596bb6489 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:21.850687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:21.851416 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:21.851847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:21.852193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] PATH_INFO: `/v3/users/6ccd91b62c2f45dcb778e50f7e8af590` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:21.853409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ccd91b62c2f45dcb778e50f7e8af590)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:21.855803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:22.080281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ccd91b62c2f45dcb778e50f7e8af590'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.080982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ccd91b62c2f45dcb778e50f7e8af590'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.273279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ccd91b62c2f45dcb778e50f7e8af590'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.281983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:22.294642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ccd91b62c2f45dcb778e50f7e8af590'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.310928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ccd91b62c2f45dcb778e50f7e8af590'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.319410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1064/2119] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:21 2022] DELETE /identity/v3/users/6ccd91b62c2f45dcb778e50f7e8af590 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:22.324320 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8429f3-eb22-4014-b78c-1dcb3e3606a0 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:22.324833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8429f3-eb22-4014-b78c-1dcb3e3606a0 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:22.325078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8429f3-eb22-4014-b78c-1dcb3e3606a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:22.357759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8a8429f3-eb22-4014-b78c-1dcb3e3606a0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:22.360063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8a8429f3-eb22-4014-b78c-1dcb3e3606a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:22.472718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:22.475110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:22.479986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1056/2120] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:22.490979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8429f3-eb22-4014-b78c-1dcb3e3606a0 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:22.493469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8a8429f3-eb22-4014-b78c-1dcb3e3606a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:22.509576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899c957-0d4a-46ae-805d-d5f5bb1981d7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:22.521532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:22.522296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:22.523438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:22.523662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] PATH_INFO: `/v3/projects/1b24364202a44dd5a936fca370419e28` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:22.525307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b24364202a44dd5a936fca370419e28)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:22.527414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:22.585835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8429f3-eb22-4014-b78c-1dcb3e3606a0 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:22.594552 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fce628-88c1-45ea-8f2a-84c50f049bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:22.595101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fce628-88c1-45ea-8f2a-84c50f049bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:22.595604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fce628-88c1-45ea-8f2a-84c50f049bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:22.596147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fce628-88c1-45ea-8f2a-84c50f049bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:22.611711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fce628-88c1-45ea-8f2a-84c50f049bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:22.614061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fce628-88c1-45ea-8f2a-84c50f049bb5 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:22.627297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1057/2121] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:22.686776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b24364202a44dd5a936fca370419e28', 'request_id': 'req-b493e933-4f1f-447d-844e-82b139b626fa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.778040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b24364202a44dd5a936fca370419e28', 'request_id': 'req-b493e933-4f1f-447d-844e-82b139b626fa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.787589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b24364202a44dd5a936fca370419e28', 'request_id': 'req-b493e933-4f1f-447d-844e-82b139b626fa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.804348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:22.806782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] The token cache is being invalidate because project 1b24364202a44dd5a936fca370419e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:22.807377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.808249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b493e933-4f1f-447d-844e-82b139b626fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:22.815250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1065/2122] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:22 2022] DELETE /identity/v3/projects/1b24364202a44dd5a936fca370419e28 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:22.817059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568c74-59a2-4e0b-85c1-6f70b92996c4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:22.817348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568c74-59a2-4e0b-85c1-6f70b92996c4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:22.817685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd568c74-59a2-4e0b-85c1-6f70b92996c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:22.842161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bd568c74-59a2-4e0b-85c1-6f70b92996c4 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:22.843408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bd568c74-59a2-4e0b-85c1-6f70b92996c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:22.915501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1058/2123] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:22.923372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41d90-91f3-4f1d-bf14-346b47b642a9 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:22.924664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41d90-91f3-4f1d-bf14-346b47b642a9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:22.926025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f41d90-91f3-4f1d-bf14-346b47b642a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:22.950831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f5f41d90-91f3-4f1d-bf14-346b47b642a9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:22.952103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f5f41d90-91f3-4f1d-bf14-346b47b642a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:22.966671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1066/2124] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:22.969898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd568c74-59a2-4e0b-85c1-6f70b92996c4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:22.981613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd568c74-59a2-4e0b-85c1-6f70b92996c4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:22.988213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:22.988896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:22.989267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:22.989637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:22.990902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:22.991366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:22.993070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:22.994614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f41d90-91f3-4f1d-bf14-346b47b642a9 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:22.996001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f5f41d90-91f3-4f1d-bf14-346b47b642a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:23.009407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f41d90-91f3-4f1d-bf14-346b47b642a9 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:23.010164 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1059/2125] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:23.011148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:23.015967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:23.016813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:23.017085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:23.017351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] PATH_INFO: `/v3/users/a496b0d69dd24d6980fa7301b7fa2a90` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:23.018531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a496b0d69dd24d6980fa7301b7fa2a90)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:23.020435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:23.025834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9caf6d8-2ecd-4ea5-9166-0faa00fec0e2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:23.030941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ff22-4dcf-4a8e-8c0d-bef6aa4b8f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:23.031863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ff22-4dcf-4a8e-8c0d-bef6aa4b8f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:23.032218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ff22-4dcf-4a8e-8c0d-bef6aa4b8f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:23.032558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529ff22-4dcf-4a8e-8c0d-bef6aa4b8f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:23.033265 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529ff22-4dcf-4a8e-8c0d-bef6aa4b8f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:23.035529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529ff22-4dcf-4a8e-8c0d-bef6aa4b8f53 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:23.123377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1060/2126] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:23 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:23.128534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ff22-4dcf-4a8e-8c0d-bef6aa4b8f53 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:23.198107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529ff22-4dcf-4a8e-8c0d-bef6aa4b8f53 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:23.203180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:23.204043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:23.204393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:23.204599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:23.205228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:23.207205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:23.227690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a496b0d69dd24d6980fa7301b7fa2a90'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.228320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a496b0d69dd24d6980fa7301b7fa2a90'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.377793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a496b0d69dd24d6980fa7301b7fa2a90'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.384915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1061/2127] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:23 2022] POST /identity/v3/users => generated 292 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:23.386705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:23.389632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:23.401733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a496b0d69dd24d6980fa7301b7fa2a90'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.404358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6d5f-f5f2-4443-a5dc-4d05b0e545af admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:23.409300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:23.409986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:23.410253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a496b0d69dd24d6980fa7301b7fa2a90'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.410479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:23.410635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] PATH_INFO: `/v3/projects/55cdd5fda2fd402d9df8a438b1c7d8cc` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:23.411790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55cdd5fda2fd402d9df8a438b1c7d8cc)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:23.413683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:23.426857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1067/2128] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:22 2022] DELETE /identity/v3/users/a496b0d69dd24d6980fa7301b7fa2a90 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:23.427851 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:23.433564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:23.524080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55cdd5fda2fd402d9df8a438b1c7d8cc', 'request_id': 'req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.525574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abadaf6-ae6f-4aae-bb7e-0b18de46b7fc admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:23.589708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:23.590371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:23.590638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:23.593350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] PATH_INFO: `/v3/users/697f296a14b44f8cbc5a3ff3c0fc18f1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:23.593350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=697f296a14b44f8cbc5a3ff3c0fc18f1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:23.594745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:23.640496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55cdd5fda2fd402d9df8a438b1c7d8cc', 'request_id': 'req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.647660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55cdd5fda2fd402d9df8a438b1c7d8cc', 'request_id': 'req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.676540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] The token cache is being invalidate because project 55cdd5fda2fd402d9df8a438b1c7d8cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:23.676828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.679666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.679666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1062/2129] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:23 2022] DELETE /identity/v3/projects/55cdd5fda2fd402d9df8a438b1c7d8cc => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:23.680457 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:23.682973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:23.698919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5fc7e-e0bb-4ef1-ad1a-bbc3b1e58e40 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:23.704535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:23.705133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:23.705414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:23.705910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:23.706772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:23.712679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:23.727474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1063/2130] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:23.727474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:23.727474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:23.740132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cdca6-2c86-4243-a619-756b89547f53 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:23.744982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:23.745595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:23.745883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:23.746139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] PATH_INFO: `/v3/users/d5d0617d66e24ec89a87f16dd10ae35b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:23.747311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d0617d66e24ec89a87f16dd10ae35b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:23.749254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:23.848580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '697f296a14b44f8cbc5a3ff3c0fc18f1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.852562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '697f296a14b44f8cbc5a3ff3c0fc18f1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.862952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:23.983280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '697f296a14b44f8cbc5a3ff3c0fc18f1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:23.998439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '697f296a14b44f8cbc5a3ff3c0fc18f1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.006371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d0617d66e24ec89a87f16dd10ae35b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.007587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '697f296a14b44f8cbc5a3ff3c0fc18f1'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.019210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1068/2131] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:23 2022] DELETE /identity/v3/users/697f296a14b44f8cbc5a3ff3c0fc18f1 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:24.019210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.019210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.087335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706c344-6528-4087-bab1-dacf3c319787 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.091973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.092573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.092852 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.093123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.093784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.095449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:24.105555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1069/2132] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:24.111483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.111483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.120352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eae9b6-5edd-48c6-97bf-cdf42fc3bee2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.125679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.126270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.126579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.126859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] PATH_INFO: `/v3/projects/b4291c4cdc1d4f4eae035c4800670101` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.128274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4291c4cdc1d4f4eae035c4800670101)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.129531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d0617d66e24ec89a87f16dd10ae35b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.130155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:24.131047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d0617d66e24ec89a87f16dd10ae35b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.143891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d0617d66e24ec89a87f16dd10ae35b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.157205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-30e62b2e-6893-43e7-8dec-230c28dcd9d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d0617d66e24ec89a87f16dd10ae35b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.163746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1064/2133] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:23 2022] DELETE /identity/v3/users/d5d0617d66e24ec89a87f16dd10ae35b => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:24.164971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28785718-821e-4f2c-8aba-1a6a6325804c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.165253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28785718-821e-4f2c-8aba-1a6a6325804c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.165522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28785718-821e-4f2c-8aba-1a6a6325804c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.185056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-28785718-821e-4f2c-8aba-1a6a6325804c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:24.186257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-28785718-821e-4f2c-8aba-1a6a6325804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.246390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1065/2134] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:24.247238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28785718-821e-4f2c-8aba-1a6a6325804c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.251830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-28785718-821e-4f2c-8aba-1a6a6325804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.263557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28785718-821e-4f2c-8aba-1a6a6325804c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.271025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4291c4cdc1d4f4eae035c4800670101', 'request_id': 'req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.332095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf535fd8-e9c4-44de-aab8-779df74a2e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.332780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf535fd8-e9c4-44de-aab8-779df74a2e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.333059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf535fd8-e9c4-44de-aab8-779df74a2e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.333332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf535fd8-e9c4-44de-aab8-779df74a2e63 admin admin] PATH_INFO: `/v3/projects/26e5450f7210459abe13107874b1d487/users/8dc859952ec94221bf85d9609d12e123/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.347237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4291c4cdc1d4f4eae035c4800670101', 'request_id': 'req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.351384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf535fd8-e9c4-44de-aab8-779df74a2e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e5450f7210459abe13107874b1d487, user_id=8dc859952ec94221bf85d9609d12e123, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.354113 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf535fd8-e9c4-44de-aab8-779df74a2e63 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:24.354995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4291c4cdc1d4f4eae035c4800670101', 'request_id': 'req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.369075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] The token cache is being invalidate because project b4291c4cdc1d4f4eae035c4800670101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:24.369438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.370149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.375332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1070/2135] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:24 2022] DELETE /identity/v3/projects/b4291c4cdc1d4f4eae035c4800670101 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:24.376490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.377849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.441397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3c21b-69ea-456c-b26e-c4b05ee6f3dd admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.446293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822bbe3-3a6a-4d21-b0b4-b558a85e7719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.449195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822bbe3-3a6a-4d21-b0b4-b558a85e7719 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.449195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822bbe3-3a6a-4d21-b0b4-b558a85e7719 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.449195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822bbe3-3a6a-4d21-b0b4-b558a85e7719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.457144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0822bbe3-3a6a-4d21-b0b4-b558a85e7719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.462442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1066/2136] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:24 2022] PUT /identity/v3/projects/26e5450f7210459abe13107874b1d487/users/8dc859952ec94221bf85d9609d12e123/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:24.471905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822bbe3-3a6a-4d21-b0b4-b558a85e7719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.472301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ac8a-37b5-4c2c-a0ab-50c6dbeebaf4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.483252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ac8a-37b5-4c2c-a0ab-50c6dbeebaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.483252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ac8a-37b5-4c2c-a0ab-50c6dbeebaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.483502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822bbe3-3a6a-4d21-b0b4-b558a85e7719 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:24.485311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1071/2137] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:24.486771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d2053-4b36-4bb1-8aeb-ef2149a6a8d9 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.487134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d2053-4b36-4bb1-8aeb-ef2149a6a8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.488298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d2053-4b36-4bb1-8aeb-ef2149a6a8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.494233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a2a1ac8a-37b5-4c2c-a0ab-50c6dbeebaf4 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:24.496998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a2a1ac8a-37b5-4c2c-a0ab-50c6dbeebaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.522893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7f6d2053-4b36-4bb1-8aeb-ef2149a6a8d9 None None] MFA Rules not processed for user `8dc859952ec94221bf85d9609d12e123`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:24.524209 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7f6d2053-4b36-4bb1-8aeb-ef2149a6a8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.559966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1067/2138] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:24.568434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48a883-50f0-46d9-90c5-2b6cecd9ec4c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.568717 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48a883-50f0-46d9-90c5-2b6cecd9ec4c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.568991 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48a883-50f0-46d9-90c5-2b6cecd9ec4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.587973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-dd48a883-50f0-46d9-90c5-2b6cecd9ec4c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:24.589158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dd48a883-50f0-46d9-90c5-2b6cecd9ec4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.590954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1072/2139] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:24.596109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d2053-4b36-4bb1-8aeb-ef2149a6a8d9 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.597818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7f6d2053-4b36-4bb1-8aeb-ef2149a6a8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.598935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1068/2140] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:24.602809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48a883-50f0-46d9-90c5-2b6cecd9ec4c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.616792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d2053-4b36-4bb1-8aeb-ef2149a6a8d9 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.620627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48a883-50f0-46d9-90c5-2b6cecd9ec4c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.627257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70595781-c31c-4354-b4ec-22ce533bac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.627828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.628203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70595781-c31c-4354-b4ec-22ce533bac14 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.628578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70595781-c31c-4354-b4ec-22ce533bac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.628687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.629009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.629144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70595781-c31c-4354-b4ec-22ce533bac14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.629374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.630077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:24.630611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70595781-c31c-4354-b4ec-22ce533bac14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.630724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.632253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:24.632413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70595781-c31c-4354-b4ec-22ce533bac14 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:24.650492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1069/2141] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:22:24.651551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.653885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.670774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1367ef-cc7c-477f-95b9-cc5de467c704 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.676448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.677065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.677338 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.677625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] PATH_INFO: `/v3/projects/b64c7734dc4c4e9186cfe80fe8470dca` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.678886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b64c7734dc4c4e9186cfe80fe8470dca)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.680790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:24.766267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1073/2142] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:24 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:24.767307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70595781-c31c-4354-b4ec-22ce533bac14 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.805866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b64c7734dc4c4e9186cfe80fe8470dca', 'request_id': 'req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.836132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70595781-c31c-4354-b4ec-22ce533bac14 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.872749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b64c7734dc4c4e9186cfe80fe8470dca', 'request_id': 'req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.881229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b64c7734dc4c4e9186cfe80fe8470dca', 'request_id': 'req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.893834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] The token cache is being invalidate because project b64c7734dc4c4e9186cfe80fe8470dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:24.894094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.894737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:24.896257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1070/2143] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:24 2022] DELETE /identity/v3/projects/b64c7734dc4c4e9186cfe80fe8470dca => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:24.898587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:24.898587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:24.905313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.906589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.906954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.907297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.908174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.910468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:24.916321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c379c-9af2-4f44-9dd8-2dd6a3a64b59 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:24.923029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:24.923808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:24.924632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:24.924937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] PATH_INFO: `/v3/users/9f801d273c7c44ecaa3cb05fcf982d0a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:24.926591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f801d273c7c44ecaa3cb05fcf982d0a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:24.929497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.025974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1074/2144] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:24 2022] POST /identity/v3/projects => generated 389 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:25.025974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.027961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.114858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59efbbe7-173c-4382-998c-9b8eec35943e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.119764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9113b0-2c61-43bf-b2b5-333a290a14a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.120403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9113b0-2c61-43bf-b2b5-333a290a14a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.120676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9113b0-2c61-43bf-b2b5-333a290a14a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.120992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9113b0-2c61-43bf-b2b5-333a290a14a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.121626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9113b0-2c61-43bf-b2b5-333a290a14a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.123666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9113b0-2c61-43bf-b2b5-333a290a14a8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.177152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f801d273c7c44ecaa3cb05fcf982d0a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.278404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1075/2145] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:25 2022] POST /identity/v3/users => generated 292 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:25.279649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc1e9-4fcf-4b3d-82ce-47e9f82b98a2 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.279929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc1e9-4fcf-4b3d-82ce-47e9f82b98a2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.280198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fc1e9-4fcf-4b3d-82ce-47e9f82b98a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.296366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:25.300373 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f801d273c7c44ecaa3cb05fcf982d0a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.300373 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f801d273c7c44ecaa3cb05fcf982d0a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.300672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-784fc1e9-4fcf-4b3d-82ce-47e9f82b98a2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:25.301875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-784fc1e9-4fcf-4b3d-82ce-47e9f82b98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.305988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f801d273c7c44ecaa3cb05fcf982d0a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.309807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1076/2146] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:25.316764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fc1e9-4fcf-4b3d-82ce-47e9f82b98a2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.316764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-784fc1e9-4fcf-4b3d-82ce-47e9f82b98a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.323947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fc1e9-4fcf-4b3d-82ce-47e9f82b98a2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.325864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f801d273c7c44ecaa3cb05fcf982d0a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.329533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.330132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.330438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.330707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.331333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.333158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1071/2147] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:24 2022] DELETE /identity/v3/users/9f801d273c7c44ecaa3cb05fcf982d0a => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:25.333297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.334212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.336060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.401883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f6e6f-3316-4ea4-bf2a-0e960ee32685 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.403529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.404148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.404499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.404668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] PATH_INFO: `/v3/projects/de3172ce471840bba0241dbebfda2a8a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.405825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de3172ce471840bba0241dbebfda2a8a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.408469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.428245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1077/2148] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:25 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:25.433205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.433205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.445352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e34f0-eac3-4c87-a298-ad0c0b9d1f8e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.450056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5cce7-36cc-4cd3-831a-742f0f531893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.450677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5cce7-36cc-4cd3-831a-742f0f531893 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.450951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5cce7-36cc-4cd3-831a-742f0f531893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.451218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5cce7-36cc-4cd3-831a-742f0f531893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.452015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5cce7-36cc-4cd3-831a-742f0f531893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.454307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5cce7-36cc-4cd3-831a-742f0f531893 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.465095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1078/2149] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:25.466217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee15e1-06e9-4b46-84e0-485fb9e73390 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.466520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee15e1-06e9-4b46-84e0-485fb9e73390 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.466786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee15e1-06e9-4b46-84e0-485fb9e73390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.487870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-69ee15e1-06e9-4b46-84e0-485fb9e73390 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:25.489070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-69ee15e1-06e9-4b46-84e0-485fb9e73390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.496873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1079/2150] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:25.498019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c2eb-8a0f-4a43-8125-00c550a9a1b7 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.498292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c2eb-8a0f-4a43-8125-00c550a9a1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.498605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c2eb-8a0f-4a43-8125-00c550a9a1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.517302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-fb73c2eb-8a0f-4a43-8125-00c550a9a1b7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:25.518553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fb73c2eb-8a0f-4a43-8125-00c550a9a1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.526040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1080/2151] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:25.526828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73c2eb-8a0f-4a43-8125-00c550a9a1b7 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.536731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73c2eb-8a0f-4a43-8125-00c550a9a1b7 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.541558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64db8f0-63dd-4f0f-9667-1bee7c03419c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.542145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64db8f0-63dd-4f0f-9667-1bee7c03419c admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.542451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64db8f0-63dd-4f0f-9667-1bee7c03419c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.542724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64db8f0-63dd-4f0f-9667-1bee7c03419c admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.543355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64db8f0-63dd-4f0f-9667-1bee7c03419c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.544935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64db8f0-63dd-4f0f-9667-1bee7c03419c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.555544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1081/2152] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:25.556167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64db8f0-63dd-4f0f-9667-1bee7c03419c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.562169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de3172ce471840bba0241dbebfda2a8a', 'request_id': 'req-266646c6-315c-475e-995f-cfe6704cdb51'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.566712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64db8f0-63dd-4f0f-9667-1bee7c03419c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.622503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b657d-c261-43d8-a9be-557efc4fdfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.623100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b657d-c261-43d8-a9be-557efc4fdfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.623376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b657d-c261-43d8-a9be-557efc4fdfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.623645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b657d-c261-43d8-a9be-557efc4fdfa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.624272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b657d-c261-43d8-a9be-557efc4fdfa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.626044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b657d-c261-43d8-a9be-557efc4fdfa2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.635450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1082/2153] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:25.636216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b657d-c261-43d8-a9be-557efc4fdfa2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.640333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de3172ce471840bba0241dbebfda2a8a', 'request_id': 'req-266646c6-315c-475e-995f-cfe6704cdb51'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.645776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b657d-c261-43d8-a9be-557efc4fdfa2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.646625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de3172ce471840bba0241dbebfda2a8a', 'request_id': 'req-266646c6-315c-475e-995f-cfe6704cdb51'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.650437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.651016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.651293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.652208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.652208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:25.652467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.654134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.659012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] The token cache is being invalidate because project de3172ce471840bba0241dbebfda2a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:25.659294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.659946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:25.661449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1072/2154] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:25 2022] DELETE /identity/v3/projects/de3172ce471840bba0241dbebfda2a8a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:25.666366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.666520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1083/2155] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:25.667399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.670977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.671104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.684363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a6891-1dd1-4076-86e1-374313e7dc7c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.689984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.690697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.690883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.691161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.691800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.693522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.705489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1084/2156] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:25.706363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.708010 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.720259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429165c0-98b0-42b8-9aa6-969694c33463 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.725101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9b2d-2efd-4546-9ed4-591c0ac9376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.725750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9b2d-2efd-4546-9ed4-591c0ac9376c admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.726029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9b2d-2efd-4546-9ed4-591c0ac9376c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.726298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9b2d-2efd-4546-9ed4-591c0ac9376c admin admin] PATH_INFO: `/v3/projects/de2ec7cdedad430cae3444cd8ec01525/users/76153c8e3951413195e180580582dffa/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.735368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266646c6-315c-475e-995f-cfe6704cdb51 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.739327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf204d2-7b06-4fe1-a251-48c47ee8f14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.740012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf204d2-7b06-4fe1-a251-48c47ee8f14a service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.740403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf204d2-7b06-4fe1-a251-48c47ee8f14a service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.740745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf204d2-7b06-4fe1-a251-48c47ee8f14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.741257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e9b2d-2efd-4546-9ed4-591c0ac9376c admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2ec7cdedad430cae3444cd8ec01525, user_id=76153c8e3951413195e180580582dffa, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.742532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dbf204d2-7b06-4fe1-a251-48c47ee8f14a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.744040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e9b2d-2efd-4546-9ed4-591c0ac9376c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.755232 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf204d2-7b06-4fe1-a251-48c47ee8f14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.756781 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf204d2-7b06-4fe1-a251-48c47ee8f14a service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.762386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:25.768415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1073/2157] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:25.769691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f1e6a-8fab-480e-a999-0a36346e35ee None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.769980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f1e6a-8fab-480e-a999-0a36346e35ee None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.770259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f1e6a-8fab-480e-a999-0a36346e35ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.791309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9e5f1e6a-8fab-480e-a999-0a36346e35ee None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:25.792523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9e5f1e6a-8fab-480e-a999-0a36346e35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.801995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1074/2158] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:25.803173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f1e6a-8fab-480e-a999-0a36346e35ee None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.804462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9e5f1e6a-8fab-480e-a999-0a36346e35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:25.819930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f1e6a-8fab-480e-a999-0a36346e35ee None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.824797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4161fe-b03d-4cfd-925a-4187509d6b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.825404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4161fe-b03d-4cfd-925a-4187509d6b70 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.825676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4161fe-b03d-4cfd-925a-4187509d6b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.825969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4161fe-b03d-4cfd-925a-4187509d6b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.826643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4161fe-b03d-4cfd-925a-4187509d6b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.828665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4161fe-b03d-4cfd-925a-4187509d6b70 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.840189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1085/2159] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:25 2022] PUT /identity/v3/projects/de2ec7cdedad430cae3444cd8ec01525/users/76153c8e3951413195e180580582dffa/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:25.841013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9b2d-2efd-4546-9ed4-591c0ac9376c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:25.905018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9b2d-2efd-4546-9ed4-591c0ac9376c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:25.910138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99945f66-2c55-4541-a833-5605744643d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:25.910821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99945f66-2c55-4541-a833-5605744643d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.911121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99945f66-2c55-4541-a833-5605744643d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.911388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99945f66-2c55-4541-a833-5605744643d1 admin admin] PATH_INFO: `/v3/projects/fbe6c3ad574e46fd9d3b37ead0542a18/users/27be0979440d4ab081dae31790280eff/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.926291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99945f66-2c55-4541-a833-5605744643d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe6c3ad574e46fd9d3b37ead0542a18, user_id=27be0979440d4ab081dae31790280eff, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:25.929381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99945f66-2c55-4541-a833-5605744643d1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:25.940938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1075/2160] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:25 2022] POST /identity/v3/projects => generated 391 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:25.940938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732ef4b-f744-4395-9c2a-b861234b127d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:25.940938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732ef4b-f744-4395-9c2a-b861234b127d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:25.940938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732ef4b-f744-4395-9c2a-b861234b127d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:25.958659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:25.959470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4732ef4b-f744-4395-9c2a-b861234b127d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:25.960661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4732ef4b-f744-4395-9c2a-b861234b127d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:26.021812 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1076/2161] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:26.024677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3e30-ddcd-41dc-b22e-e10f29857c8e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.024677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3e30-ddcd-41dc-b22e-e10f29857c8e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.024677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3e30-ddcd-41dc-b22e-e10f29857c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.053215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1086/2162] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:25 2022] PUT /identity/v3/projects/fbe6c3ad574e46fd9d3b37ead0542a18/users/27be0979440d4ab081dae31790280eff/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:26.054009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b90c3e30-ddcd-41dc-b22e-e10f29857c8e None None] MFA Rules not processed for user `76153c8e3951413195e180580582dffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:26.054098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99945f66-2c55-4541-a833-5605744643d1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.054170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b90c3e30-ddcd-41dc-b22e-e10f29857c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:26.132105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1077/2163] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:26.133080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c3e30-ddcd-41dc-b22e-e10f29857c8e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.153167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99945f66-2c55-4541-a833-5605744643d1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.167073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ddd6f-4463-4e3a-91ac-cad745d03a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.167699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ddd6f-4463-4e3a-91ac-cad745d03a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.168013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ddd6f-4463-4e3a-91ac-cad745d03a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.168369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ddd6f-4463-4e3a-91ac-cad745d03a88 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.168907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ddd6f-4463-4e3a-91ac-cad745d03a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.170859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ddd6f-4463-4e3a-91ac-cad745d03a88 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.230121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c3e30-ddcd-41dc-b22e-e10f29857c8e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.236476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54380d0-7bb4-4b87-97a0-810648b6d2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.237154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54380d0-7bb4-4b87-97a0-810648b6d2dc admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.237505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54380d0-7bb4-4b87-97a0-810648b6d2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.237839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54380d0-7bb4-4b87-97a0-810648b6d2dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.238590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54380d0-7bb4-4b87-97a0-810648b6d2dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:26.238953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54380d0-7bb4-4b87-97a0-810648b6d2dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.240730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54380d0-7bb4-4b87-97a0-810648b6d2dc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.256618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1078/2164] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:26.258093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c45bd-e5f4-4631-96fa-b0ca00729950 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.258593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c45bd-e5f4-4631-96fa-b0ca00729950 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.258985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55c45bd-e5f4-4631-96fa-b0ca00729950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.278379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a55c45bd-e5f4-4631-96fa-b0ca00729950 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:26.279762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a55c45bd-e5f4-4631-96fa-b0ca00729950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:26.287701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1079/2165] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:26.290839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c45bd-e5f4-4631-96fa-b0ca00729950 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.298632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55c45bd-e5f4-4631-96fa-b0ca00729950 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.303516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f37e3-10db-429f-9224-2dbd142cf4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.304119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f37e3-10db-429f-9224-2dbd142cf4ba admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.304392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f37e3-10db-429f-9224-2dbd142cf4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.304659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f37e3-10db-429f-9224-2dbd142cf4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.305282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f37e3-10db-429f-9224-2dbd142cf4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.307243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f37e3-10db-429f-9224-2dbd142cf4ba admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.316726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1087/2166] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:26 2022] POST /identity/v3/users => generated 296 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:26.317797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ddd6f-4463-4e3a-91ac-cad745d03a88 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.334185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ddd6f-4463-4e3a-91ac-cad745d03a88 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.334185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.334185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.334530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.334626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.335186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.337062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.447714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1088/2167] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:26 2022] POST /identity/v3/projects => generated 369 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:26.450831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.451003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1080/2168] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:26 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:26.451003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f37e3-10db-429f-9224-2dbd142cf4ba admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.451377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:26.514459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf34839-1479-439e-8251-697952b54e48 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.515327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f37e3-10db-429f-9224-2dbd142cf4ba admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.520352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.520953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.521225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.521491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.522132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.523754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.534423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1081/2169] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:26.535268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.536974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:26.549983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dde77-a558-40fe-b223-126d02245e3d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.554610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.555216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.555488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.555751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] PATH_INFO: `/v3/projects/75569f93065d4dca9fc67911dd786792` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.556867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=75569f93065d4dca9fc67911dd786792)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.558793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.570163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78c35d-6dad-4293-a307-6d7e1931bfcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.573845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c35d-6dad-4293-a307-6d7e1931bfcb service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.573845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c35d-6dad-4293-a307-6d7e1931bfcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.573845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78c35d-6dad-4293-a307-6d7e1931bfcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.580695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78c35d-6dad-4293-a307-6d7e1931bfcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.582247 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78c35d-6dad-4293-a307-6d7e1931bfcb service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.591902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1089/2170] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:26.596012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78c35d-6dad-4293-a307-6d7e1931bfcb service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.603088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78c35d-6dad-4293-a307-6d7e1931bfcb service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.608057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c8b2d-e339-48a5-8164-46a267249953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.608762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8b2d-e339-48a5-8164-46a267249953 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.609045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8b2d-e339-48a5-8164-46a267249953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.609957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8b2d-e339-48a5-8164-46a267249953 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.610621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c8b2d-e339-48a5-8164-46a267249953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.612587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c8b2d-e339-48a5-8164-46a267249953 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.661537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75569f93065d4dca9fc67911dd786792', 'request_id': 'req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:26.730026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1090/2171] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:26 2022] POST /identity/v3/users => generated 285 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:26.730026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c8b2d-e339-48a5-8164-46a267249953 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.738544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c8b2d-e339-48a5-8164-46a267249953 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.748975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75569f93065d4dca9fc67911dd786792', 'request_id': 'req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:26.756141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75569f93065d4dca9fc67911dd786792', 'request_id': 'req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:26.771602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] The token cache is being invalidate because project 75569f93065d4dca9fc67911dd786792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:26.771939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:26.772623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:26.786174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1082/2172] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:26 2022] DELETE /identity/v3/projects/75569f93065d4dca9fc67911dd786792 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:26.786174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.786174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:26.856794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764971-db9d-460f-93d4-f8127f4f7329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.857695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dc4f9-2e03-439f-a48f-4213d9ddf6cc admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.857981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764971-db9d-460f-93d4-f8127f4f7329 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.858329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764971-db9d-460f-93d4-f8127f4f7329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.858691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93764971-db9d-460f-93d4-f8127f4f7329 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.860012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93764971-db9d-460f-93d4-f8127f4f7329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.862619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93764971-db9d-460f-93d4-f8127f4f7329 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.863527 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.864242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.864597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.864928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.865656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.867540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.879681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1083/2173] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:26.880695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.882817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:26.898530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a742b-44ef-4761-a182-d0db6d5ae5e4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.903996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742f9b9-83d3-4ef3-bf87-4c5f545f24c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.904625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742f9b9-83d3-4ef3-bf87-4c5f545f24c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.905193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742f9b9-83d3-4ef3-bf87-4c5f545f24c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.905469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742f9b9-83d3-4ef3-bf87-4c5f545f24c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.906755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742f9b9-83d3-4ef3-bf87-4c5f545f24c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.908627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742f9b9-83d3-4ef3-bf87-4c5f545f24c6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.920735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1084/2174] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:26.921933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742f9b9-83d3-4ef3-bf87-4c5f545f24c6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.933469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742f9b9-83d3-4ef3-bf87-4c5f545f24c6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.939495 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a2a7e-e15e-4026-b0d3-323979f36203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.940175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a2a7e-e15e-4026-b0d3-323979f36203 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.940524 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a2a7e-e15e-4026-b0d3-323979f36203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.940797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a2a7e-e15e-4026-b0d3-323979f36203 admin admin] PATH_INFO: `/v3/projects/679eaf5f4f42419f9995d4e426c924a3/users/580d61e2ad3d48d5870b7cffb8f6359c/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.960125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1091/2175] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:26 2022] POST /identity/v3/users => generated 299 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:26.960125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764971-db9d-460f-93d4-f8127f4f7329 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:26.969946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a2a7e-e15e-4026-b0d3-323979f36203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=679eaf5f4f42419f9995d4e426c924a3, user_id=580d61e2ad3d48d5870b7cffb8f6359c, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.973592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a2a7e-e15e-4026-b0d3-323979f36203 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:26.981422 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93764971-db9d-460f-93d4-f8127f4f7329 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:26.987943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:26.992338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:26.992338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:26.992338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:26.992338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:26.992338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.004194 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1092/2176] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:27.005217 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.007379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:27.020866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75128c13-ad10-47f8-b4fa-9d923cfe29f2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.025718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb167ea7-99e8-4265-9409-0dc339d30ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.026318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167ea7-99e8-4265-9409-0dc339d30ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.026627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167ea7-99e8-4265-9409-0dc339d30ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.026915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb167ea7-99e8-4265-9409-0dc339d30ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.027539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb167ea7-99e8-4265-9409-0dc339d30ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.029221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb167ea7-99e8-4265-9409-0dc339d30ae5 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.040418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1093/2177] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:27.041288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb167ea7-99e8-4265-9409-0dc339d30ae5 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.050544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1085/2178] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:26 2022] PUT /identity/v3/projects/679eaf5f4f42419f9995d4e426c924a3/users/580d61e2ad3d48d5870b7cffb8f6359c/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:27.053289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb167ea7-99e8-4265-9409-0dc339d30ae5 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.061013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a2a7e-e15e-4026-b0d3-323979f36203 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.111677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c283665-e1c6-4376-87c8-03e7d048f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.112338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c283665-e1c6-4376-87c8-03e7d048f94e admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.112614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c283665-e1c6-4376-87c8-03e7d048f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.112889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c283665-e1c6-4376-87c8-03e7d048f94e admin admin] PATH_INFO: `/v3/projects/f10308bd1a0e49a58e079801bbb3121d/users/f53c3be0d0034e118ad5d5b4e2899a59/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.121759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a2a7e-e15e-4026-b0d3-323979f36203 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.128321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c283665-e1c6-4376-87c8-03e7d048f94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10308bd1a0e49a58e079801bbb3121d, user_id=f53c3be0d0034e118ad5d5b4e2899a59, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.128911 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b78e2-433f-4a2c-afee-ed56f6596b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.129514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b78e2-433f-4a2c-afee-ed56f6596b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.129788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b78e2-433f-4a2c-afee-ed56f6596b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.130064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b78e2-433f-4a2c-afee-ed56f6596b68 admin admin] PATH_INFO: `/v3/projects/3e7e7c6727564c70a565bc1d13df5240/users/6ef863a067fb464f8d1db48de205b295/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.131083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c283665-e1c6-4376-87c8-03e7d048f94e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.146855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b78e2-433f-4a2c-afee-ed56f6596b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e7e7c6727564c70a565bc1d13df5240, user_id=6ef863a067fb464f8d1db48de205b295, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.149637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3b78e2-433f-4a2c-afee-ed56f6596b68 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.268244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1094/2179] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:27 2022] PUT /identity/v3/projects/f10308bd1a0e49a58e079801bbb3121d/users/f53c3be0d0034e118ad5d5b4e2899a59/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:27.270711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93637d-490b-4fe9-9470-80d79da58502 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.271056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93637d-490b-4fe9-9470-80d79da58502 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.271324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93637d-490b-4fe9-9470-80d79da58502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.304808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2f93637d-490b-4fe9-9470-80d79da58502 None None] MFA Rules not processed for user `580d61e2ad3d48d5870b7cffb8f6359c`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:27.305679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1086/2180] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:27 2022] PUT /identity/v3/projects/3e7e7c6727564c70a565bc1d13df5240/users/6ef863a067fb464f8d1db48de205b295/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:27.306331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2f93637d-490b-4fe9-9470-80d79da58502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:27.309547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5cb1a-464a-4efe-9c17-eba232c24f2b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.309834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5cb1a-464a-4efe-9c17-eba232c24f2b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.310317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5cb1a-464a-4efe-9c17-eba232c24f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.343379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3df5cb1a-464a-4efe-9c17-eba232c24f2b None None] MFA Rules not processed for user `f53c3be0d0034e118ad5d5b4e2899a59`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:27.345675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3df5cb1a-464a-4efe-9c17-eba232c24f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:27.382286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1095/2181] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:27.386749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93637d-490b-4fe9-9470-80d79da58502 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.433163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1087/2182] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:27.433956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5cb1a-464a-4efe-9c17-eba232c24f2b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.454809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:27.508466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93637d-490b-4fe9-9470-80d79da58502 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.515135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.517979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.518271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.518569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.520410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.522436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.531975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5cb1a-464a-4efe-9c17-eba232c24f2b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.535533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:27.539225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e78fc-e5ca-42ce-b0cb-7390c95efae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.540179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e78fc-e5ca-42ce-b0cb-7390c95efae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.540638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e78fc-e5ca-42ce-b0cb-7390c95efae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.541078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e78fc-e5ca-42ce-b0cb-7390c95efae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.542026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e78fc-e5ca-42ce-b0cb-7390c95efae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.545004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e78fc-e5ca-42ce-b0cb-7390c95efae7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.550517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1096/2183] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:27.551477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.555444 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:27.623719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50d87f-b378-4c82-997b-e9651a005328 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.628322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994969cc-02f4-4d88-8961-ce93b94a2103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.628992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994969cc-02f4-4d88-8961-ce93b94a2103 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.629262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994969cc-02f4-4d88-8961-ce93b94a2103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.629526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994969cc-02f4-4d88-8961-ce93b94a2103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.641572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994969cc-02f4-4d88-8961-ce93b94a2103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.644234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994969cc-02f4-4d88-8961-ce93b94a2103 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.660594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1088/2184] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:27 2022] POST /identity/v3/projects => generated 393 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:27.660594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e78fc-e5ca-42ce-b0cb-7390c95efae7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.664742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1097/2185] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:27.666180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994969cc-02f4-4d88-8961-ce93b94a2103 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.693340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994969cc-02f4-4d88-8961-ce93b94a2103 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.769122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930645d9-c6c2-42aa-9776-2f668d6fd37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.770058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930645d9-c6c2-42aa-9776-2f668d6fd37a admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.770679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930645d9-c6c2-42aa-9776-2f668d6fd37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.771342 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930645d9-c6c2-42aa-9776-2f668d6fd37a admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.772265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930645d9-c6c2-42aa-9776-2f668d6fd37a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.775679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930645d9-c6c2-42aa-9776-2f668d6fd37a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.875457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1098/2186] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:27 2022] POST /identity/v3/users => generated 303 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:27.877186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930645d9-c6c2-42aa-9776-2f668d6fd37a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:27.901796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e78fc-e5ca-42ce-b0cb-7390c95efae7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.905483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930645d9-c6c2-42aa-9776-2f668d6fd37a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:27.909882 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d883070-dac8-4f8d-9df0-6abadb317c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.911010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d883070-dac8-4f8d-9df0-6abadb317c26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.911470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d883070-dac8-4f8d-9df0-6abadb317c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.911900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d883070-dac8-4f8d-9df0-6abadb317c26 admin admin] PATH_INFO: `/v3/projects/3e7e7c6727564c70a565bc1d13df5240/users/6ef863a067fb464f8d1db48de205b295/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.914430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:27.915649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d883070-dac8-4f8d-9df0-6abadb317c26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e7e7c6727564c70a565bc1d13df5240, user_id=6ef863a067fb464f8d1db48de205b295, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.915764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:27.915764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:27.915764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] PATH_INFO: `/v3/users/f53c3be0d0034e118ad5d5b4e2899a59` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:27.917006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53c3be0d0034e118ad5d5b4e2899a59)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:27.918336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d883070-dac8-4f8d-9df0-6abadb317c26 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.919038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:27.933894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1089/2187] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:27 2022] PUT /identity/v3/projects/3e7e7c6727564c70a565bc1d13df5240/users/6ef863a067fb464f8d1db48de205b295/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:27.934825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d883070-dac8-4f8d-9df0-6abadb317c26 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:28.008339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d883070-dac8-4f8d-9df0-6abadb317c26 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:28.013039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b81fa2-00ac-4cbe-92fc-0edc1d20de93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:28.013643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81fa2-00ac-4cbe-92fc-0edc1d20de93 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.013913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81fa2-00ac-4cbe-92fc-0edc1d20de93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.014176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81fa2-00ac-4cbe-92fc-0edc1d20de93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.014937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b81fa2-00ac-4cbe-92fc-0edc1d20de93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:28.016905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b81fa2-00ac-4cbe-92fc-0edc1d20de93 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:28.027425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1090/2188] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:28.028707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de656f-0271-4ebd-b2d8-96121a44ce90 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.029065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de656f-0271-4ebd-b2d8-96121a44ce90 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.029329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de656f-0271-4ebd-b2d8-96121a44ce90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.061439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d3de656f-0271-4ebd-b2d8-96121a44ce90 None None] MFA Rules not processed for user `6ef863a067fb464f8d1db48de205b295`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:28.062994 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d3de656f-0271-4ebd-b2d8-96121a44ce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:28.087434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:28.144004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1091/2189] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:28.148381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de656f-0271-4ebd-b2d8-96121a44ce90 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:28.162810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de656f-0271-4ebd-b2d8-96121a44ce90 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:28.168867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53c3be0d0034e118ad5d5b4e2899a59'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:28.178631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:28.178925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53c3be0d0034e118ad5d5b4e2899a59'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:28.179038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.179038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.179038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] PATH_INFO: `/v3/projects/36360397e7084751b4a4c0231e5fae71/users/d1bcbfe61e3f40098a45e9b4fccd6711/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.191937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=36360397e7084751b4a4c0231e5fae71, user_id=d1bcbfe61e3f40098a45e9b4fccd6711, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:28.195274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:28.314090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53c3be0d0034e118ad5d5b4e2899a59'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:28.323213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1092/2190] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:28 2022] PUT /identity/v3/projects/36360397e7084751b4a4c0231e5fae71/users/d1bcbfe61e3f40098a45e9b4fccd6711/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:28.324197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:28.330820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:28.342876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53c3be0d0034e118ad5d5b4e2899a59'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:28.353389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53c3be0d0034e118ad5d5b4e2899a59'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:28.363419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1099/2191] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:27 2022] DELETE /identity/v3/users/f53c3be0d0034e118ad5d5b4e2899a59 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:28.364506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:28.365731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:28.428734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df153c4-882e-40d9-8972-b126fe5b88ff admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:28.452358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb280a94-d2bc-4527-a622-ed384c1f8519 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:28.457631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7361b63-5c87-4550-bead-0e6cc677f84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:28.458243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7361b63-5c87-4550-bead-0e6cc677f84d admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.458548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7361b63-5c87-4550-bead-0e6cc677f84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.458824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7361b63-5c87-4550-bead-0e6cc677f84d admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.459515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7361b63-5c87-4550-bead-0e6cc677f84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:28.461308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7361b63-5c87-4550-bead-0e6cc677f84d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:28.471855 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1100/2192] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:28.476344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24686d5-122b-45f7-bdc6-03c110e6ee20 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.477087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24686d5-122b-45f7-bdc6-03c110e6ee20 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.477510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24686d5-122b-45f7-bdc6-03c110e6ee20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.486987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773bb58-4252-46ed-88c7-918d6edf986c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:28.487595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773bb58-4252-46ed-88c7-918d6edf986c service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.487880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773bb58-4252-46ed-88c7-918d6edf986c service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.488452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773bb58-4252-46ed-88c7-918d6edf986c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.489843 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5773bb58-4252-46ed-88c7-918d6edf986c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:28.501080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f24686d5-122b-45f7-bdc6-03c110e6ee20 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:28.502583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f24686d5-122b-45f7-bdc6-03c110e6ee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:28.511362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1101/2193] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:28.512249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24686d5-122b-45f7-bdc6-03c110e6ee20 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:28.524632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24686d5-122b-45f7-bdc6-03c110e6ee20 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:28.533270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:28.534645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.535025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.538483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] PATH_INFO: `/v3/projects/36360397e7084751b4a4c0231e5fae71/users/d1bcbfe61e3f40098a45e9b4fccd6711/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.538483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36360397e7084751b4a4c0231e5fae71, user_id=d1bcbfe61e3f40098a45e9b4fccd6711, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:28.541316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:28.558426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1102/2194] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:28 2022] PUT /identity/v3/projects/36360397e7084751b4a4c0231e5fae71/users/d1bcbfe61e3f40098a45e9b4fccd6711/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:28.558426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:28.565659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:28.581828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773bb58-4252-46ed-88c7-918d6edf986c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:28.583575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773bb58-4252-46ed-88c7-918d6edf986c service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:28.607735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1093/2195] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:28.608834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe336229-9f10-4a82-8628-e594820e502f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.609788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe336229-9f10-4a82-8628-e594820e502f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.609986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe336229-9f10-4a82-8628-e594820e502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.661780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fe336229-9f10-4a82-8628-e594820e502f None None] MFA Rules not processed for user `d1bcbfe61e3f40098a45e9b4fccd6711`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:28.662838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe4cfb-e4a4-4ed4-90e6-1af7d071e264 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:28.663281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fe336229-9f10-4a82-8628-e594820e502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:28.666721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5c282-3c40-48a0-af2f-dbd50c468496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:28.667340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5c282-3c40-48a0-af2f-dbd50c468496 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.667624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5c282-3c40-48a0-af2f-dbd50c468496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.667898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5c282-3c40-48a0-af2f-dbd50c468496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.723366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:28.783104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5c282-3c40-48a0-af2f-dbd50c468496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:28.786238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5c282-3c40-48a0-af2f-dbd50c468496 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:28.805346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1094/2196] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:28.811558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1103/2197] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:28.812735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:28.822720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe336229-9f10-4a82-8628-e594820e502f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:28.825898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fe336229-9f10-4a82-8628-e594820e502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:28.899794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe336229-9f10-4a82-8628-e594820e502f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:28.918371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:28.919675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:28.920134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:28.920835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:28.954561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:28.956170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:28.969121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1095/2198] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:28 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:29.028052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5c282-3c40-48a0-af2f-dbd50c468496 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:29.037607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5c282-3c40-48a0-af2f-dbd50c468496 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:29.041232 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:29.041838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:29.042111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:29.042403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:29.044203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:29.131957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:29.134521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:29.161523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1104/2199] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:29.438811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:29.440275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:29.458064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8918086d-0c2a-472e-a4ff-3763f9d2faa4 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:29.466711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:29.467872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:29.468086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:29.468350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] PATH_INFO: `/v3/projects/f10308bd1a0e49a58e079801bbb3121d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:29.472531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10308bd1a0e49a58e079801bbb3121d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:29.474795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:29.613190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10308bd1a0e49a58e079801bbb3121d', 'request_id': 'req-9ba818fa-7c86-4820-956e-57fd14894cdc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:29.717739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10308bd1a0e49a58e079801bbb3121d', 'request_id': 'req-9ba818fa-7c86-4820-956e-57fd14894cdc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:29.725505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10308bd1a0e49a58e079801bbb3121d', 'request_id': 'req-9ba818fa-7c86-4820-956e-57fd14894cdc'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:29.738246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] The token cache is being invalidate because project f10308bd1a0e49a58e079801bbb3121d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:29.738575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:29.739221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:29.740721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1096/2200] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:29 2022] DELETE /identity/v3/projects/f10308bd1a0e49a58e079801bbb3121d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:29.753002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:30.146469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:30.847563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:30.856887 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92e57d-8c11-451c-ae9a-1b4b1c7c321a service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:30.923903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b60cc-d866-4227-9644-cbff2e78d9fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:30.924774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b60cc-d866-4227-9644-cbff2e78d9fd service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:30.925327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b60cc-d866-4227-9644-cbff2e78d9fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:30.925843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b60cc-d866-4227-9644-cbff2e78d9fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:30.928913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f53b60cc-d866-4227-9644-cbff2e78d9fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:30.992322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b60cc-d866-4227-9644-cbff2e78d9fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:30.993919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b60cc-d866-4227-9644-cbff2e78d9fd service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:31.003617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1105/2201] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:31.012333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:31.659242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:32.950547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:32.953014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:32.970869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba818fa-7c86-4820-956e-57fd14894cdc admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:32.975315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:32.976241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:32.976701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:32.977144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:32.979237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:33.117501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:33.119345 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:33.135722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1097/2202] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:33.187883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c0c3f-93e6-4d76-9534-5db8ef08e1cc None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:33.188439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c0c3f-93e6-4d76-9534-5db8ef08e1cc None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:33.188439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c0c3f-93e6-4d76-9534-5db8ef08e1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:33.240220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ef8c0c3f-93e6-4d76-9534-5db8ef08e1cc None None] MFA Rules not processed for user `27be0979440d4ab081dae31790280eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:33.241538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ef8c0c3f-93e6-4d76-9534-5db8ef08e1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:33.323987 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1106/2203] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:33.334607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:33.336150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:33.430505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6f768-858d-4c4b-892f-18e9944ef562 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:33.436598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59840ea2-f3ba-4c6a-b7f0-9e2ef3524c9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:33.437234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59840ea2-f3ba-4c6a-b7f0-9e2ef3524c9c service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:33.437523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59840ea2-f3ba-4c6a-b7f0-9e2ef3524c9c service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:33.437815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59840ea2-f3ba-4c6a-b7f0-9e2ef3524c9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:33.449566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59840ea2-f3ba-4c6a-b7f0-9e2ef3524c9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:33.451546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59840ea2-f3ba-4c6a-b7f0-9e2ef3524c9c service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:33.465793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1098/2204] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:34.096250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:34.610173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:35.129391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c0c3f-93e6-4d76-9534-5db8ef08e1cc None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:35.136302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ef8c0c3f-93e6-4d76-9534-5db8ef08e1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:35.201738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c0c3f-93e6-4d76-9534-5db8ef08e1cc None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:35.208045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:35.208713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:35.209062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:35.209407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] PATH_INFO: `/v3/users/580d61e2ad3d48d5870b7cffb8f6359c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:35.210666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=580d61e2ad3d48d5870b7cffb8f6359c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:35.212816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:35.462764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '580d61e2ad3d48d5870b7cffb8f6359c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:35.463532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '580d61e2ad3d48d5870b7cffb8f6359c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:35.572463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '580d61e2ad3d48d5870b7cffb8f6359c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:35.588565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '580d61e2ad3d48d5870b7cffb8f6359c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:35.607011 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '580d61e2ad3d48d5870b7cffb8f6359c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:35.614301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1107/2205] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:35 2022] DELETE /identity/v3/users/580d61e2ad3d48d5870b7cffb8f6359c => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:35.618803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccdbd1-78d7-49aa-bf46-619dcfa8f875 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:35.625704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccdbd1-78d7-49aa-bf46-619dcfa8f875 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:35.625704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccdbd1-78d7-49aa-bf46-619dcfa8f875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:35.639982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-56ccdbd1-78d7-49aa-bf46-619dcfa8f875 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:35.641303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-56ccdbd1-78d7-49aa-bf46-619dcfa8f875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:35.708043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1099/2206] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:35.718853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:35.720643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:35.929659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507eea18-d393-404d-ba8b-e09c50a530b2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:35.937466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:35.938117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:35.938415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:35.938625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:35.959734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:35.962554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:35.975614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1108/2207] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:36.350011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9d57f-6173-443d-9f24-fdfb61fe1d49 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:36.350370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9d57f-6173-443d-9f24-fdfb61fe1d49 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:36.350647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9d57f-6173-443d-9f24-fdfb61fe1d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:36.376822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a2c9d57f-6173-443d-9f24-fdfb61fe1d49 None None] MFA Rules not processed for user `27be0979440d4ab081dae31790280eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:36.378315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a2c9d57f-6173-443d-9f24-fdfb61fe1d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:36.433624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:36.456816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1100/2208] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:36.461288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:36.464643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:36.467578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:36.503525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84939303-74d4-4a7c-9a2b-d797bd3d0f26 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:36.508816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:36.509556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:36.509950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:36.510318 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] PATH_INFO: `/v3/projects/679eaf5f4f42419f9995d4e426c924a3` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:36.511662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] RBAC: Authorizing `identity:delete_project(project_id=679eaf5f4f42419f9995d4e426c924a3)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:36.513641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:36.630719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '679eaf5f4f42419f9995d4e426c924a3', 'request_id': 'req-21a88622-ed82-4d28-a546-d905198065df'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:36.725238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '679eaf5f4f42419f9995d4e426c924a3', 'request_id': 'req-21a88622-ed82-4d28-a546-d905198065df'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:36.733594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '679eaf5f4f42419f9995d4e426c924a3', 'request_id': 'req-21a88622-ed82-4d28-a546-d905198065df'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:36.753612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] The token cache is being invalidate because project 679eaf5f4f42419f9995d4e426c924a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:36.754226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:36.763173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:36.782068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1109/2209] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:36 2022] DELETE /identity/v3/projects/679eaf5f4f42419f9995d4e426c924a3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:36.790506 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9d57f-6173-443d-9f24-fdfb61fe1d49 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:36.807892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9d57f-6173-443d-9f24-fdfb61fe1d49 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:36.971569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:36.972538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:36.972976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:36.973402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] PATH_INFO: `/v3/users/d1bcbfe61e3f40098a45e9b4fccd6711` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:36.975185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1bcbfe61e3f40098a45e9b4fccd6711)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:36.978329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:37.210626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1bcbfe61e3f40098a45e9b4fccd6711'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:37.318288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1bcbfe61e3f40098a45e9b4fccd6711'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:37.322271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1bcbfe61e3f40098a45e9b4fccd6711'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:37.327022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1bcbfe61e3f40098a45e9b4fccd6711'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:37.345738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-efbc413e-0e34-4881-a411-284d40f43b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1bcbfe61e3f40098a45e9b4fccd6711'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:37.352572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1101/2210] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:36 2022] DELETE /identity/v3/users/d1bcbfe61e3f40098a45e9b4fccd6711 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:38.031688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:38.033478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:38.052322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a88622-ed82-4d28-a546-d905198065df admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:38.113595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:38.114218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:38.114527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:38.114802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] PATH_INFO: `/v3/projects/36360397e7084751b4a4c0231e5fae71` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:38.115913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36360397e7084751b4a4c0231e5fae71)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:38.117820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:38.267462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36360397e7084751b4a4c0231e5fae71', 'request_id': 'req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:38.394748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36360397e7084751b4a4c0231e5fae71', 'request_id': 'req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:38.400797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36360397e7084751b4a4c0231e5fae71', 'request_id': 'req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:38.407518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:38.419848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] The token cache is being invalidate because project 36360397e7084751b4a4c0231e5fae71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:38.420143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:38.420816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ea7a31c0-f9ec-4a23-ae84-d38a20ee9d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:38.423465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1110/2211] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:38 2022] DELETE /identity/v3/projects/36360397e7084751b4a4c0231e5fae71 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:38.428037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560a1b8-1391-4e7d-860f-042e3fa60b50 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:38.428322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560a1b8-1391-4e7d-860f-042e3fa60b50 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:38.428620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560a1b8-1391-4e7d-860f-042e3fa60b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:38.448497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8560a1b8-1391-4e7d-860f-042e3fa60b50 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:38.449744 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8560a1b8-1391-4e7d-860f-042e3fa60b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:38.495518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:38.511179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1102/2212] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:38.518860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a74ecc-c3e6-42ff-958f-3393e4c08f49 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:38.519078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a74ecc-c3e6-42ff-958f-3393e4c08f49 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:38.519375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a74ecc-c3e6-42ff-958f-3393e4c08f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:38.539720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-60a74ecc-c3e6-42ff-958f-3393e4c08f49 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:38.541037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-60a74ecc-c3e6-42ff-958f-3393e4c08f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:38.550148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1111/2213] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:38.554445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560a1b8-1391-4e7d-860f-042e3fa60b50 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:38.567512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560a1b8-1391-4e7d-860f-042e3fa60b50 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:38.573184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:38.573806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:38.574063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:38.574664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:38.575043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:38.575335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:38.577105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:38.589329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1103/2214] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:22:38.594180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a74ecc-c3e6-42ff-958f-3393e4c08f49 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:38.603867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a74ecc-c3e6-42ff-958f-3393e4c08f49 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:38.608936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751d5e7-dc49-416f-86f9-a0bfaf7f0a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:38.611048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751d5e7-dc49-416f-86f9-a0bfaf7f0a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:38.611327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751d5e7-dc49-416f-86f9-a0bfaf7f0a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:38.611591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751d5e7-dc49-416f-86f9-a0bfaf7f0a64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:38.612232 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751d5e7-dc49-416f-86f9-a0bfaf7f0a64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:38.621853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1751d5e7-dc49-416f-86f9-a0bfaf7f0a64 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:38.698311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:38.700163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:38.714598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c5ce-83e9-4609-ace0-0db9c9cc22c2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:38.720319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:38.720953 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:38.721224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:38.721489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] PATH_INFO: `/v3/users/8dc859952ec94221bf85d9609d12e123` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:38.722668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dc859952ec94221bf85d9609d12e123)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:38.724656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:38.762191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1112/2215] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:38 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:22:38.766953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751d5e7-dc49-416f-86f9-a0bfaf7f0a64 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:38.842942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751d5e7-dc49-416f-86f9-a0bfaf7f0a64 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:38.856045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f2a7b-8fa7-40be-994c-9bb2e452c537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:38.856685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f2a7b-8fa7-40be-994c-9bb2e452c537 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:38.856958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f2a7b-8fa7-40be-994c-9bb2e452c537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:38.857227 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f2a7b-8fa7-40be-994c-9bb2e452c537 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:38.857867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f2a7b-8fa7-40be-994c-9bb2e452c537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:38.860639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f2a7b-8fa7-40be-994c-9bb2e452c537 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:38.958694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dc859952ec94221bf85d9609d12e123'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:38.993165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1113/2216] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:38 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:38.997110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f2a7b-8fa7-40be-994c-9bb2e452c537 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:39.008939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f2a7b-8fa7-40be-994c-9bb2e452c537 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:39.016765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ada3b-0258-4283-86ad-2c200af5cf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:39.020200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ada3b-0258-4283-86ad-2c200af5cf43 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:39.020200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ada3b-0258-4283-86ad-2c200af5cf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:39.020200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ada3b-0258-4283-86ad-2c200af5cf43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:39.020200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ada3b-0258-4283-86ad-2c200af5cf43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:39.020889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ada3b-0258-4283-86ad-2c200af5cf43 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:39.031969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1114/2217] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:22:39.036124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ada3b-0258-4283-86ad-2c200af5cf43 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:39.047306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ada3b-0258-4283-86ad-2c200af5cf43 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:39.052235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8c31c-34c9-410f-a0d3-7957ddfb0816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:39.052554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8c31c-34c9-410f-a0d3-7957ddfb0816 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:39.052828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8c31c-34c9-410f-a0d3-7957ddfb0816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:39.053093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8c31c-34c9-410f-a0d3-7957ddfb0816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:39.053750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8c31c-34c9-410f-a0d3-7957ddfb0816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:39.055371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8c31c-34c9-410f-a0d3-7957ddfb0816 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:39.071359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1115/2218] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:22:39.072631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8c31c-34c9-410f-a0d3-7957ddfb0816 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:39.073547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dc859952ec94221bf85d9609d12e123'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:39.074225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dc859952ec94221bf85d9609d12e123'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:39.083216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dc859952ec94221bf85d9609d12e123'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:39.086903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8c31c-34c9-410f-a0d3-7957ddfb0816 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:39.088407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2691a0f-01ce-48c5-b40e-97e563a9e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:39.089257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2691a0f-01ce-48c5-b40e-97e563a9e6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:39.089525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2691a0f-01ce-48c5-b40e-97e563a9e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:39.089794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2691a0f-01ce-48c5-b40e-97e563a9e6ef admin admin] PATH_INFO: `/v3/projects/29e14e807cde40918ebbe16fcc0aa333/users/f7ae4c1d809849cfbde738963b992774/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:39.098649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-1ceb3354-571a-49e2-b7b4-0a609d7b60a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dc859952ec94221bf85d9609d12e123'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:39.106530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1104/2219] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:38 2022] DELETE /identity/v3/users/8dc859952ec94221bf85d9609d12e123 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:39.111473 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2691a0f-01ce-48c5-b40e-97e563a9e6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e14e807cde40918ebbe16fcc0aa333, user_id=f7ae4c1d809849cfbde738963b992774, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:39.111610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4292966-8011-4a5a-a2da-3b45fca4f0eb None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:39.117938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4292966-8011-4a5a-a2da-3b45fca4f0eb None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:39.117938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4292966-8011-4a5a-a2da-3b45fca4f0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:39.118132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2691a0f-01ce-48c5-b40e-97e563a9e6ef admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:39.132022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a4292966-8011-4a5a-a2da-3b45fca4f0eb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:39.133236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a4292966-8011-4a5a-a2da-3b45fca4f0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:39.191240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1105/2220] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:39.201202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4292966-8011-4a5a-a2da-3b45fca4f0eb None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:39.202627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a4292966-8011-4a5a-a2da-3b45fca4f0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:39.246658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1116/2221] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:39 2022] PUT /identity/v3/projects/29e14e807cde40918ebbe16fcc0aa333/users/f7ae4c1d809849cfbde738963b992774/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:39.258307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916b8c-fbcc-4b4e-8bfa-40710cb38cae None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:39.258629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916b8c-fbcc-4b4e-8bfa-40710cb38cae None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:39.258942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916b8c-fbcc-4b4e-8bfa-40710cb38cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:39.263442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4292966-8011-4a5a-a2da-3b45fca4f0eb None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:39.289961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-42916b8c-fbcc-4b4e-8bfa-40710cb38cae None None] MFA Rules not processed for user `f7ae4c1d809849cfbde738963b992774`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:39.291503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-42916b8c-fbcc-4b4e-8bfa-40710cb38cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:39.327981 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:39.328885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:39.329154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:39.329426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:39.367947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1117/2222] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:39.380364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42916b8c-fbcc-4b4e-8bfa-40710cb38cae None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:39.381827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-42916b8c-fbcc-4b4e-8bfa-40710cb38cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:39.392289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:39.393818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:39.404199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1106/2223] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:39.472927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42916b8c-fbcc-4b4e-8bfa-40710cb38cae None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:39.476660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51044b-cb6e-4ddf-9a15-3116cf172adf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1842c116caa4a5d9d75c8605eeec0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:39.477256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51044b-cb6e-4ddf-9a15-3116cf172adf service glance] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:39.477525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51044b-cb6e-4ddf-9a15-3116cf172adf service glance] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:39.477790 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51044b-cb6e-4ddf-9a15-3116cf172adf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:39.487572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51044b-cb6e-4ddf-9a15-3116cf172adf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:39.488994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51044b-cb6e-4ddf-9a15-3116cf172adf service glance] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:39.498438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1118/2224] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:39.918833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:39.920235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:39.933726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca2b42-3bc1-46ae-8762-c6d4e2cc8ce1 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:39.940381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:39.941780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:39.941780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:39.942047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] PATH_INFO: `/v3/projects/26e5450f7210459abe13107874b1d487` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:39.943407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e5450f7210459abe13107874b1d487)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:39.946419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:39.989868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:40.089973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e5450f7210459abe13107874b1d487', 'request_id': 'req-4199e1bc-96f2-41f9-9057-811959eaf093'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.195357 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e5450f7210459abe13107874b1d487', 'request_id': 'req-4199e1bc-96f2-41f9-9057-811959eaf093'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.201398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e5450f7210459abe13107874b1d487', 'request_id': 'req-4199e1bc-96f2-41f9-9057-811959eaf093'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.213531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] The token cache is being invalidate because project 26e5450f7210459abe13107874b1d487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:40.213820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.214491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.215967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1107/2225] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:39 2022] DELETE /identity/v3/projects/26e5450f7210459abe13107874b1d487 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:40.220096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51044b-cb6e-4ddf-9a15-3116cf172adf service glance] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:40.230987 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51044b-cb6e-4ddf-9a15-3116cf172adf service glance] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:40.286590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:40.287200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:40.287503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:40.287777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] PATH_INFO: `/v3/users/76153c8e3951413195e180580582dffa` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:40.288865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=76153c8e3951413195e180580582dffa)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:40.290787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:40.492788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76153c8e3951413195e180580582dffa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.493455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76153c8e3951413195e180580582dffa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.582065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76153c8e3951413195e180580582dffa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.594793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:40.604468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76153c8e3951413195e180580582dffa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.614785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76153c8e3951413195e180580582dffa'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:40.621098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1119/2226] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:40 2022] DELETE /identity/v3/users/76153c8e3951413195e180580582dffa => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:41.176608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:41.178215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:41.185074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:41.208794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199e1bc-96f2-41f9-9057-811959eaf093 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:41.308950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:41.311118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:41.311450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:41.311715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] PATH_INFO: `/v3/projects/de2ec7cdedad430cae3444cd8ec01525` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:41.313818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2ec7cdedad430cae3444cd8ec01525)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:41.315155 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:41.493298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2ec7cdedad430cae3444cd8ec01525', 'request_id': 'req-08bb258a-3f86-4c41-b421-e801ea7fa775'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:41.595480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2ec7cdedad430cae3444cd8ec01525', 'request_id': 'req-08bb258a-3f86-4c41-b421-e801ea7fa775'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:41.607072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2ec7cdedad430cae3444cd8ec01525', 'request_id': 'req-08bb258a-3f86-4c41-b421-e801ea7fa775'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:41.619124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] The token cache is being invalidate because project de2ec7cdedad430cae3444cd8ec01525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:41.619446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:41.620106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:41.621598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1108/2227] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:41 2022] DELETE /identity/v3/projects/de2ec7cdedad430cae3444cd8ec01525 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:41.626690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:41.628165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:41.641240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0b51a-1a30-446c-8413-ce97fe140a9f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:41.706721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:41.707386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:41.707892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:41.707997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] PATH_INFO: `/v3/users/6ef863a067fb464f8d1db48de205b295` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:41.709116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ef863a067fb464f8d1db48de205b295)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:41.711088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:41.932616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ef863a067fb464f8d1db48de205b295'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:41.934046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ef863a067fb464f8d1db48de205b295'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.007641 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ef863a067fb464f8d1db48de205b295'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.022416 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ef863a067fb464f8d1db48de205b295'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.029768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-ed36e9f4-c9fc-4149-9d32-c431d1fb98e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ef863a067fb464f8d1db48de205b295'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.036795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1120/2228] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:22:41 2022] DELETE /identity/v3/users/6ef863a067fb464f8d1db48de205b295 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:22:42.241525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:42.305289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:42.514295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:42.516026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:42.529134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb258a-3f86-4c41-b421-e801ea7fa775 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:42.605516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:42.606393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:42.606574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:42.606955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] PATH_INFO: `/v3/projects/3e7e7c6727564c70a565bc1d13df5240` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:42.608271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e7e7c6727564c70a565bc1d13df5240)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:42.610877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:42.749612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e7e7c6727564c70a565bc1d13df5240', 'request_id': 'req-afba2053-8a1d-4a7f-aa8b-227e92929fbf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.818124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e7e7c6727564c70a565bc1d13df5240', 'request_id': 'req-afba2053-8a1d-4a7f-aa8b-227e92929fbf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.820856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:42.826256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e7e7c6727564c70a565bc1d13df5240', 'request_id': 'req-afba2053-8a1d-4a7f-aa8b-227e92929fbf'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.846230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] The token cache is being invalidate because project 3e7e7c6727564c70a565bc1d13df5240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:22:42.846934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.848171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-afba2053-8a1d-4a7f-aa8b-227e92929fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:22:42.854224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1109/2229] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:22:42 2022] DELETE /identity/v3/projects/3e7e7c6727564c70a565bc1d13df5240 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:42.856067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ea0df-5c81-4ccc-b603-2859273cf612 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:42.856522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ea0df-5c81-4ccc-b603-2859273cf612 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:42.856802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ea0df-5c81-4ccc-b603-2859273cf612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:42.882227 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-433ea0df-5c81-4ccc-b603-2859273cf612 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:42.883502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-433ea0df-5c81-4ccc-b603-2859273cf612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:42.895326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:42.946494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1121/2230] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:22:42.954786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a2db-5d55-46e3-9130-940896eda4dd None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:42.955073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a2db-5d55-46e3-9130-940896eda4dd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:42.955348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a2db-5d55-46e3-9130-940896eda4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:42.979274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-47c0a2db-5d55-46e3-9130-940896eda4dd None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:42.980908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-47c0a2db-5d55-46e3-9130-940896eda4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:42.991168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1110/2231] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:42.995560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ea0df-5c81-4ccc-b603-2859273cf612 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:43.005793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433ea0df-5c81-4ccc-b603-2859273cf612 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:43.012400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:43.013051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:43.013330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:43.013597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:43.014248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:22:43.015704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:43.017345 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:43.029889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1122/2232] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:22:43.044520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0a2db-5d55-46e3-9130-940896eda4dd None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:43.052847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0a2db-5d55-46e3-9130-940896eda4dd None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:43.057948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28280e-a2c2-4488-a352-4c8b0848e27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:43.058256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28280e-a2c2-4488-a352-4c8b0848e27d admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:43.058557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28280e-a2c2-4488-a352-4c8b0848e27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:43.058860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28280e-a2c2-4488-a352-4c8b0848e27d admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:43.059522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28280e-a2c2-4488-a352-4c8b0848e27d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:43.062321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28280e-a2c2-4488-a352-4c8b0848e27d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:43.141680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1111/2233] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:22:43 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:43.146147 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:43.157456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5665b3-1967-48e0-87ba-2056d6d32398 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:43.218613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2573e7-3f83-4e4c-93a4-f8294afb801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:43.219262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2573e7-3f83-4e4c-93a4-f8294afb801e admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:43.219582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2573e7-3f83-4e4c-93a4-f8294afb801e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:43.222771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2573e7-3f83-4e4c-93a4-f8294afb801e admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:43.222771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2573e7-3f83-4e4c-93a4-f8294afb801e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:43.222771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2573e7-3f83-4e4c-93a4-f8294afb801e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:43.347377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1123/2234] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:22:43 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:22:43.351665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28280e-a2c2-4488-a352-4c8b0848e27d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:43.363842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28280e-a2c2-4488-a352-4c8b0848e27d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:43.368819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6da130-6408-4f84-8265-51494af45b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:43.369540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6da130-6408-4f84-8265-51494af45b71 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:43.369685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6da130-6408-4f84-8265-51494af45b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:43.369955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6da130-6408-4f84-8265-51494af45b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:43.370636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6da130-6408-4f84-8265-51494af45b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:43.372240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6da130-6408-4f84-8265-51494af45b71 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:43.383676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1112/2235] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:43.388063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2573e7-3f83-4e4c-93a4-f8294afb801e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:43.400200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2573e7-3f83-4e4c-93a4-f8294afb801e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:43.405136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc4adc-1a04-40e8-b14c-bcf9d5440020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:43.405730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4adc-1a04-40e8-b14c-bcf9d5440020 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:43.406033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4adc-1a04-40e8-b14c-bcf9d5440020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:43.406324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4adc-1a04-40e8-b14c-bcf9d5440020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:43.406969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc4adc-1a04-40e8-b14c-bcf9d5440020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:43.408568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc4adc-1a04-40e8-b14c-bcf9d5440020 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:43.418541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1124/2236] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:22:43.423640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6da130-6408-4f84-8265-51494af45b71 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:43.433676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6da130-6408-4f84-8265-51494af45b71 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:43.438551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:43.439199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:43.439459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:43.439722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] PATH_INFO: `/v3/projects/44a4181be7e6404487bd7eddca1f6d8c/users/75c27b297062483598cf8ff5185e2f2a/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:43.447083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:43.455759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a4181be7e6404487bd7eddca1f6d8c, user_id=75c27b297062483598cf8ff5185e2f2a, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:43.458544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:43.555596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1113/2237] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:22:43 2022] PUT /identity/v3/projects/44a4181be7e6404487bd7eddca1f6d8c/users/75c27b297062483598cf8ff5185e2f2a/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:22:43.560664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d92331-818b-4bb3-9a23-b33c72675b19 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:43.560944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d92331-818b-4bb3-9a23-b33c72675b19 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:43.561211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d92331-818b-4bb3-9a23-b33c72675b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:43.581608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e2d92331-818b-4bb3-9a23-b33c72675b19 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:43.582891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e2d92331-818b-4bb3-9a23-b33c72675b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:43.642817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1125/2238] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:43.649302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:43.653473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:43.658277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:43.713572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258debfb-79a0-4309-b51e-d75bde00cd8d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:43.717310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:43.717904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:43.718176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:43.718521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:43.728317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:43.729749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:43.735751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:43.742064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1114/2239] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:22:43.869479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:44.072198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:44.395458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:44.809991 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:45.143391 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298880ab-c8f9-431c-89f9-f794c7b619ed None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:45.143913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298880ab-c8f9-431c-89f9-f794c7b619ed None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:45.143990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298880ab-c8f9-431c-89f9-f794c7b619ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:45.162786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:45.179618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-298880ab-c8f9-431c-89f9-f794c7b619ed None None] MFA Rules not processed for user `75c27b297062483598cf8ff5185e2f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:45.181036 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-298880ab-c8f9-431c-89f9-f794c7b619ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:45.258546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1126/2240] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:45.269760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:22:45.271665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:45.341651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c22737-f34c-4b00-8943-e206542ab512 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:22:45.347310 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:22:45.348103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:45.349616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:45.349616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:45.362848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:22:45.364730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:22:45.378742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1115/2241] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:22:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:22:45.403840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:45.982758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:46.730020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:47.072228 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:47.136957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ef25-02a3-418c-927c-6aeed7e33b9b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:22:47.137238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ef25-02a3-418c-927c-6aeed7e33b9b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:22:47.137509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421ef25-02a3-418c-927c-6aeed7e33b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:22:47.157246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7421ef25-02a3-418c-927c-6aeed7e33b9b None None] MFA Rules not processed for user `75c27b297062483598cf8ff5185e2f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:22:47.158469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7421ef25-02a3-418c-927c-6aeed7e33b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:22:47.165337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1127/2242] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:22:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:22:47.255236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:47.443663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:47.501461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=78,keystone:INSERT=5,keystone:DELETE=17 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:48.081278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:48.172947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:48.372555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:48.466913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:48.926647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:49.162358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:49.540825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:49.623765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:49.806609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:49.852545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:49.903851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:50.149430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:50.180574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:50.395380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:50.552383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:50.643893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:50.929741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:51.167533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:51.415733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:51.626516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:51.737880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:51.763468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:51.886534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:51.915023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:51.994886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=82,keystone:INSERT=5,keystone:DELETE=13 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.052530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.158285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.318226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.534812 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.620386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.706134 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.734787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.875741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:52.903966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.007200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.029825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.160768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.349707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.557815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.740875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.880976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.903822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:53.926667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.003567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.086738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.175765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.246305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.269260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.395439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.418251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.538198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.564937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.775437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:22:54.861803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:03.303432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:03.305057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:03.319651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd26f3-07f6-486a-8d37-72d0a9333dc5 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:03.324449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:03.325072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:03.325347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:03.325637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] PATH_INFO: `/v3/users/f7ae4c1d809849cfbde738963b992774` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:03.326771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ae4c1d809849cfbde738963b992774)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:03.328674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:03.504896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ae4c1d809849cfbde738963b992774'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:03.587214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ae4c1d809849cfbde738963b992774'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:03.588216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ae4c1d809849cfbde738963b992774'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:03.598333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ae4c1d809849cfbde738963b992774'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:03.617383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ae4c1d809849cfbde738963b992774'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:03.623842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1116/2243] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:23:03 2022] DELETE /identity/v3/users/f7ae4c1d809849cfbde738963b992774 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:23:03.628747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f9e3e-f33a-4aeb-a79d-2f20f85bf4ab None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:03.633579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f9e3e-f33a-4aeb-a79d-2f20f85bf4ab None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:03.633579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f9e3e-f33a-4aeb-a79d-2f20f85bf4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:03.649009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-8e5f9e3e-f33a-4aeb-a79d-2f20f85bf4ab None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:03.650411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8e5f9e3e-f33a-4aeb-a79d-2f20f85bf4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:03.710922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1128/2244] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:03.720731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:03.722210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:03.744250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:03.781756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29cb1e6-2178-4ec7-bbeb-b195016cab3f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:03.786361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fafb83-da4b-4c86-9d2b-66b4dae2cef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:03.786992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafb83-da4b-4c86-9d2b-66b4dae2cef8 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:03.787268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafb83-da4b-4c86-9d2b-66b4dae2cef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:03.787538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafb83-da4b-4c86-9d2b-66b4dae2cef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:03.796889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fafb83-da4b-4c86-9d2b-66b4dae2cef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:03.798850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fafb83-da4b-4c86-9d2b-66b4dae2cef8 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:03.808815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1117/2245] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:23:04.423351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f9e3e-f33a-4aeb-a79d-2f20f85bf4ab None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:04.424964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8e5f9e3e-f33a-4aeb-a79d-2f20f85bf4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:04.440068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f9e3e-f33a-4aeb-a79d-2f20f85bf4ab None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:04.444805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:04.445617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:04.445895 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:04.446160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] PATH_INFO: `/v3/projects/29e14e807cde40918ebbe16fcc0aa333` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:04.447328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e14e807cde40918ebbe16fcc0aa333)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:04.449204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:04.568042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e14e807cde40918ebbe16fcc0aa333', 'request_id': 'req-b2e8ec99-54eb-4f24-9039-b704d1e96061'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:04.646666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e14e807cde40918ebbe16fcc0aa333', 'request_id': 'req-b2e8ec99-54eb-4f24-9039-b704d1e96061'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:04.655529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e14e807cde40918ebbe16fcc0aa333', 'request_id': 'req-b2e8ec99-54eb-4f24-9039-b704d1e96061'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:04.687695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] The token cache is being invalidate because project 29e14e807cde40918ebbe16fcc0aa333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:23:04.688214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:04.690242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b2e8ec99-54eb-4f24-9039-b704d1e96061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:04.691992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1129/2246] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:23:04 2022] DELETE /identity/v3/projects/29e14e807cde40918ebbe16fcc0aa333 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:23:04.697004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a020-2e63-4420-8f38-38ba9cd84455 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:04.697975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a020-2e63-4420-8f38-38ba9cd84455 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:04.698315 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223a020-2e63-4420-8f38-38ba9cd84455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:04.722660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b223a020-2e63-4420-8f38-38ba9cd84455 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:04.723997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b223a020-2e63-4420-8f38-38ba9cd84455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:04.818419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1118/2247] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:04.825728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a80129-35ec-45c7-8abd-1dd533d50971 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:04.826213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a80129-35ec-45c7-8abd-1dd533d50971 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:04.826513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a80129-35ec-45c7-8abd-1dd533d50971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:04.847115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-15a80129-35ec-45c7-8abd-1dd533d50971 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:04.848603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-15a80129-35ec-45c7-8abd-1dd533d50971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:04.857054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1130/2248] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:04.861334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223a020-2e63-4420-8f38-38ba9cd84455 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:04.871694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223a020-2e63-4420-8f38-38ba9cd84455 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:04.877661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:04.878257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:04.878567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:04.878834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:04.879523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:23:04.879823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:04.881438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:04.894891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1119/2249] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:23:04.903714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a80129-35ec-45c7-8abd-1dd533d50971 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:04.913267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a80129-35ec-45c7-8abd-1dd533d50971 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:04.917879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48867719-08bb-497b-93ed-a2676903d98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:04.918510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48867719-08bb-497b-93ed-a2676903d98b admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:04.923633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48867719-08bb-497b-93ed-a2676903d98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:04.923633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48867719-08bb-497b-93ed-a2676903d98b admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:04.923633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48867719-08bb-497b-93ed-a2676903d98b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:04.923633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48867719-08bb-497b-93ed-a2676903d98b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:05.046717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1131/2250] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:23:04 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:23:05.047597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:05.059305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384db57-eb6e-469c-bbf3-8aef39a6ca70 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:05.125333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e0494-da27-4d04-be4e-e8b8812e1297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:05.125948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e0494-da27-4d04-be4e-e8b8812e1297 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:05.126913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e0494-da27-4d04-be4e-e8b8812e1297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:05.127191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e0494-da27-4d04-be4e-e8b8812e1297 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:05.127831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e0494-da27-4d04-be4e-e8b8812e1297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:05.129936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825e0494-da27-4d04-be4e-e8b8812e1297 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:05.209968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1120/2251] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:23:05 2022] POST /identity/v3/users => generated 283 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:23:05.214069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48867719-08bb-497b-93ed-a2676903d98b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:05.224998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48867719-08bb-497b-93ed-a2676903d98b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:05.229636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a198-11be-47f6-92b0-2d58d431a52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:05.230467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a198-11be-47f6-92b0-2d58d431a52c admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:05.231000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a198-11be-47f6-92b0-2d58d431a52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:05.231295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a198-11be-47f6-92b0-2d58d431a52c admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:05.231933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911a198-11be-47f6-92b0-2d58d431a52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:05.233529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911a198-11be-47f6-92b0-2d58d431a52c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:05.245617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1132/2252] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:23:05.249853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e0494-da27-4d04-be4e-e8b8812e1297 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:05.262210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e0494-da27-4d04-be4e-e8b8812e1297 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:05.267221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295bc8e-001b-4ab9-ab4f-f6e700ee7403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:05.267919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295bc8e-001b-4ab9-ab4f-f6e700ee7403 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:05.268201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295bc8e-001b-4ab9-ab4f-f6e700ee7403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:05.268474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295bc8e-001b-4ab9-ab4f-f6e700ee7403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:05.269207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295bc8e-001b-4ab9-ab4f-f6e700ee7403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:05.270793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295bc8e-001b-4ab9-ab4f-f6e700ee7403 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:05.284755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1121/2253] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:23:05.289138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a198-11be-47f6-92b0-2d58d431a52c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:05.301008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a198-11be-47f6-92b0-2d58d431a52c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:05.305805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:05.306491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:05.306769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:05.307041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] PATH_INFO: `/v3/projects/39eee6506dcb4ea892520b040eb0095d/users/4fbce4f6a4c0447fa41ce560244f7785/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:05.324922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eee6506dcb4ea892520b040eb0095d, user_id=4fbce4f6a4c0447fa41ce560244f7785, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:05.328268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:05.332749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:05.408706 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1133/2254] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:23:05 2022] PUT /identity/v3/projects/39eee6506dcb4ea892520b040eb0095d/users/4fbce4f6a4c0447fa41ce560244f7785/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:23:05.421171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f959fa-c70a-4c06-bf18-1685b85ef1f5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:05.422263 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f959fa-c70a-4c06-bf18-1685b85ef1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:05.422672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f959fa-c70a-4c06-bf18-1685b85ef1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:05.456948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f3f959fa-c70a-4c06-bf18-1685b85ef1f5 None None] MFA Rules not processed for user `4fbce4f6a4c0447fa41ce560244f7785`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:05.458320 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f3f959fa-c70a-4c06-bf18-1685b85ef1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:05.473311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:05.533389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1122/2255] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:05.545241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:05.547046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:05.607384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe9e852-d008-47cf-9159-31b07b9b66a5 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:05.611131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64251d3e-dbaa-48f9-8e38-89f742af1ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:05.611818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251d3e-dbaa-48f9-8e38-89f742af1ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:05.612169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251d3e-dbaa-48f9-8e38-89f742af1ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:05.612577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251d3e-dbaa-48f9-8e38-89f742af1ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:05.622150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64251d3e-dbaa-48f9-8e38-89f742af1ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:05.626713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64251d3e-dbaa-48f9-8e38-89f742af1ec0 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:05.633734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1134/2256] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:23:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:23:10.487387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b635b6-ef4f-4105-9a6e-3fa4c9af2d6c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:10.487930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b635b6-ef4f-4105-9a6e-3fa4c9af2d6c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:10.487930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b635b6-ef4f-4105-9a6e-3fa4c9af2d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:10.507933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d9b635b6-ef4f-4105-9a6e-3fa4c9af2d6c None None] MFA Rules not processed for user `987d2f6bb1ff4f1fa248c3ff410c8119`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:10.509127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d9b635b6-ef4f-4105-9a6e-3fa4c9af2d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:10.567694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1123/2257] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:15.629539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:20.563634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=45 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:20.601580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df632-7603-4066-a190-4d1c8210e641 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:20.602067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df632-7603-4066-a190-4d1c8210e641 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:20.602165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df632-7603-4066-a190-4d1c8210e641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:20.622103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-cb5df632-7603-4066-a190-4d1c8210e641 None None] MFA Rules not processed for user `75c27b297062483598cf8ff5185e2f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:20.623407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cb5df632-7603-4066-a190-4d1c8210e641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:20.679148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1135/2258] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:28.270692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b023d22-bc61-4bcd-954c-04609c1eaf76 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:28.271071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b023d22-bc61-4bcd-954c-04609c1eaf76 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:28.271268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b023d22-bc61-4bcd-954c-04609c1eaf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:28.296685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-1b023d22-bc61-4bcd-954c-04609c1eaf76 None None] MFA Rules not processed for user `72467431e40c491cad192b06be494175`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:28.298064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1b023d22-bc61-4bcd-954c-04609c1eaf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:28.375221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1124/2259] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:30.672350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=25 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:38.360267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=25 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:51.149732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c81d4f-43e6-43c8-b067-89a0d5240942 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:51.150219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c81d4f-43e6-43c8-b067-89a0d5240942 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:51.150328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c81d4f-43e6-43c8-b067-89a0d5240942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:51.170142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-42c81d4f-43e6-43c8-b067-89a0d5240942 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:51.171361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-42c81d4f-43e6-43c8-b067-89a0d5240942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:51.227219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1136/2260] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:23:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:51.273546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b023d22-bc61-4bcd-954c-04609c1eaf76 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:51.274966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1b023d22-bc61-4bcd-954c-04609c1eaf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:51.351056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b023d22-bc61-4bcd-954c-04609c1eaf76 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:51.357207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e1f76-f141-4eb3-8303-90a899c010fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:51.357962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e1f76-f141-4eb3-8303-90a899c010fb service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:51.358085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e1f76-f141-4eb3-8303-90a899c010fb service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:51.358374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e1f76-f141-4eb3-8303-90a899c010fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:51.367106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e1f76-f141-4eb3-8303-90a899c010fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:51.369807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e1f76-f141-4eb3-8303-90a899c010fb service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:51.377980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1125/2261] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:23:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:23:55.510918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c81d4f-43e6-43c8-b067-89a0d5240942 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:55.592238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c81d4f-43e6-43c8-b067-89a0d5240942 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:55.597453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:55.598086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:55.598472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:55.598742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] PATH_INFO: `/v3/users/4fbce4f6a4c0447fa41ce560244f7785` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:55.599959 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fbce4f6a4c0447fa41ce560244f7785)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:55.601874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:55.833226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fbce4f6a4c0447fa41ce560244f7785'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:55.834049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fbce4f6a4c0447fa41ce560244f7785'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:55.922924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fbce4f6a4c0447fa41ce560244f7785'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:55.939066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fbce4f6a4c0447fa41ce560244f7785'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:55.949683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fbce4f6a4c0447fa41ce560244f7785'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:55.957379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1137/2262] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:23:55 2022] DELETE /identity/v3/users/4fbce4f6a4c0447fa41ce560244f7785 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:23:55.961603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d92c8f-09f3-4a1f-8a8d-e25728ff131b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:55.962815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d92c8f-09f3-4a1f-8a8d-e25728ff131b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:55.963898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d92c8f-09f3-4a1f-8a8d-e25728ff131b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:55.988135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-74d92c8f-09f3-4a1f-8a8d-e25728ff131b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:55.990144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-74d92c8f-09f3-4a1f-8a8d-e25728ff131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:56.073823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1126/2263] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:23:56.085020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:56.087130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:56.133910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:56.174878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c249412-2c95-4afc-8648-52f6286fa0d7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:56.179547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c48e55-418c-4ee9-bcf1-dde61b609774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:56.180673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c48e55-418c-4ee9-bcf1-dde61b609774 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:56.180958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c48e55-418c-4ee9-bcf1-dde61b609774 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:56.181359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c48e55-418c-4ee9-bcf1-dde61b609774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:56.200020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c48e55-418c-4ee9-bcf1-dde61b609774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:56.202597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c48e55-418c-4ee9-bcf1-dde61b609774 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:56.221092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1138/2264] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:23:56.777158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d92c8f-09f3-4a1f-8a8d-e25728ff131b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:56.778834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-74d92c8f-09f3-4a1f-8a8d-e25728ff131b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:56.797210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d92c8f-09f3-4a1f-8a8d-e25728ff131b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:56.802822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:56.803575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:56.803922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:56.804270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] PATH_INFO: `/v3/projects/39eee6506dcb4ea892520b040eb0095d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:56.805479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39eee6506dcb4ea892520b040eb0095d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:56.808314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:56.944366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39eee6506dcb4ea892520b040eb0095d', 'request_id': 'req-846eb959-e690-4efa-ab30-d947820bc99b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:57.025435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39eee6506dcb4ea892520b040eb0095d', 'request_id': 'req-846eb959-e690-4efa-ab30-d947820bc99b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:57.038118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39eee6506dcb4ea892520b040eb0095d', 'request_id': 'req-846eb959-e690-4efa-ab30-d947820bc99b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:57.064427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] The token cache is being invalidate because project 39eee6506dcb4ea892520b040eb0095d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:23:57.065065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:57.066566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-846eb959-e690-4efa-ab30-d947820bc99b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:23:57.072244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1127/2265] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:23:56 2022] DELETE /identity/v3/projects/39eee6506dcb4ea892520b040eb0095d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:23:57.074907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa5219-8f69-4ce5-ab94-5fea2ee4cb9b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.076292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa5219-8f69-4ce5-ab94-5fea2ee4cb9b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.076619 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa5219-8f69-4ce5-ab94-5fea2ee4cb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.099354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e2aa5219-8f69-4ce5-ab94-5fea2ee4cb9b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:57.104436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e2aa5219-8f69-4ce5-ab94-5fea2ee4cb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:57.192737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1139/2266] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:57.200656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14970e-f859-4970-826a-87d9c3c39d39 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.200920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14970e-f859-4970-826a-87d9c3c39d39 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.201197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14970e-f859-4970-826a-87d9c3c39d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.228561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8f14970e-f859-4970-826a-87d9c3c39d39 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:57.231532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8f14970e-f859-4970-826a-87d9c3c39d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:57.243112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1128/2267] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:57.247604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa5219-8f69-4ce5-ab94-5fea2ee4cb9b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:57.259126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa5219-8f69-4ce5-ab94-5fea2ee4cb9b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:57.264499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:57.265112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.265388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.265657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.266303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:23:57.266635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:57.268222 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:57.283987 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1140/2268] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:23:57.287720 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14970e-f859-4970-826a-87d9c3c39d39 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:57.300593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14970e-f859-4970-826a-87d9c3c39d39 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:57.306255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3883fa-ffe6-41c3-95ac-0a3e7309609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:57.307179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3883fa-ffe6-41c3-95ac-0a3e7309609a admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.307653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3883fa-ffe6-41c3-95ac-0a3e7309609a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.307925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3883fa-ffe6-41c3-95ac-0a3e7309609a admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.308559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3883fa-ffe6-41c3-95ac-0a3e7309609a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:57.310622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3883fa-ffe6-41c3-95ac-0a3e7309609a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:57.315929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:57.393482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1129/2269] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:23:57 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:23:57.397862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:57.428407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130c2cc-03f8-4afd-9512-c9fe62d92d60 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:57.503534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe5933-8788-4b8e-8bff-7e75771a9628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:57.504057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe5933-8788-4b8e-8bff-7e75771a9628 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.504477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe5933-8788-4b8e-8bff-7e75771a9628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.504755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe5933-8788-4b8e-8bff-7e75771a9628 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.505399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe5933-8788-4b8e-8bff-7e75771a9628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:57.507428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe5933-8788-4b8e-8bff-7e75771a9628 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:57.638394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1141/2270] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:23:57 2022] POST /identity/v3/users => generated 287 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:23:57.641711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3883fa-ffe6-41c3-95ac-0a3e7309609a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:57.653207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3883fa-ffe6-41c3-95ac-0a3e7309609a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:57.658645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139a94e-e868-4574-a3cb-33308fe97736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:57.659273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139a94e-e868-4574-a3cb-33308fe97736 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.659549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139a94e-e868-4574-a3cb-33308fe97736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.659820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139a94e-e868-4574-a3cb-33308fe97736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.660510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139a94e-e868-4574-a3cb-33308fe97736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:57.662161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139a94e-e868-4574-a3cb-33308fe97736 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:57.678399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1130/2271] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:23:57.678606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe5933-8788-4b8e-8bff-7e75771a9628 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:57.702107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe5933-8788-4b8e-8bff-7e75771a9628 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:57.709546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be97a08-5e26-41c9-a6ca-10febd4812d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:57.710208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97a08-5e26-41c9-a6ca-10febd4812d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.710660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97a08-5e26-41c9-a6ca-10febd4812d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.710950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be97a08-5e26-41c9-a6ca-10febd4812d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.711591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be97a08-5e26-41c9-a6ca-10febd4812d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:57.714027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be97a08-5e26-41c9-a6ca-10febd4812d2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:57.739667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1142/2272] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:23:57.744014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139a94e-e868-4574-a3cb-33308fe97736 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:57.755439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139a94e-e868-4574-a3cb-33308fe97736 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:57.760288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:57.760979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.761258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.761608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] PATH_INFO: `/v3/projects/d0643dd7b0eb44938b350c31235b0e0c/users/b6cdf250c47e469a85e923a0ab7f15b3/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.781536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0643dd7b0eb44938b350c31235b0e0c, user_id=b6cdf250c47e469a85e923a0ab7f15b3, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:57.784497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:57.883122 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1131/2273] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:23:57 2022] PUT /identity/v3/projects/d0643dd7b0eb44938b350c31235b0e0c/users/b6cdf250c47e469a85e923a0ab7f15b3/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:23:57.894644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50657-53a4-4d97-8c30-61b61e1b1e0e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:57.894931 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50657-53a4-4d97-8c30-61b61e1b1e0e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:57.895204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c50657-53a4-4d97-8c30-61b61e1b1e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:57.923694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-89c50657-53a4-4d97-8c30-61b61e1b1e0e None None] MFA Rules not processed for user `b6cdf250c47e469a85e923a0ab7f15b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:23:57.926529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-89c50657-53a4-4d97-8c30-61b61e1b1e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:57.953546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:23:58.006604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1143/2274] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:23:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:23:58.018325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:23:58.027887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:23:58.105613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3beb48-4112-4176-bda3-7a45b83e7b87 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:23:58.109832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccffcd-db94-4416-b192-eb07b835049b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:23:58.110744 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccffcd-db94-4416-b192-eb07b835049b service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:23:58.110899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccffcd-db94-4416-b192-eb07b835049b service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:23:58.111392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccffcd-db94-4416-b192-eb07b835049b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:23:58.122145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccffcd-db94-4416-b192-eb07b835049b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:23:58.123899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccffcd-db94-4416-b192-eb07b835049b service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:23:58.138142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1132/2275] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:23:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:24:08.129871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:08.214439 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=17 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:13.598268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c50657-53a4-4d97-8c30-61b61e1b1e0e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:13.599854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-89c50657-53a4-4d97-8c30-61b61e1b1e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:13.672467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c50657-53a4-4d97-8c30-61b61e1b1e0e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:13.677371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:13.677968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:13.678240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:13.678535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:13.680232 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:13.692178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:13.693663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:13.704092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1144/2276] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:24:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:24:14.257313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3b894-b25f-4d5e-9e95-8d04f8cab503 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:14.257773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3b894-b25f-4d5e-9e95-8d04f8cab503 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:14.257876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e3b894-b25f-4d5e-9e95-8d04f8cab503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:14.277821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-16e3b894-b25f-4d5e-9e95-8d04f8cab503 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:14.279061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-16e3b894-b25f-4d5e-9e95-8d04f8cab503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:14.334004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1133/2277] 10.222.0.20 () {58 vars in 967 bytes} [Fri Sep 9 23:24:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:14.346791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:14.348189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:14.417952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a27b4-2aa2-4bfc-bf31-268056e3a32b service placement] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:14.423072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983a249-232b-43be-b60e-00341afb43c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:14.423616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a249-232b-43be-b60e-00341afb43c5 service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:14.423891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a249-232b-43be-b60e-00341afb43c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:14.424160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a249-232b-43be-b60e-00341afb43c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:14.432250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983a249-232b-43be-b60e-00341afb43c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:14.433798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983a249-232b-43be-b60e-00341afb43c5 service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:14.441993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1145/2278] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:24:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:24:14.491796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3b894-b25f-4d5e-9e95-8d04f8cab503 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:14.559909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e3b894-b25f-4d5e-9e95-8d04f8cab503 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:14.565127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:14.567052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:14.567340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:14.567634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] PATH_INFO: `/v3/users/b6cdf250c47e469a85e923a0ab7f15b3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:14.568966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6cdf250c47e469a85e923a0ab7f15b3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:14.570834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:14.794261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6cdf250c47e469a85e923a0ab7f15b3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:14.854218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6cdf250c47e469a85e923a0ab7f15b3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:14.854867 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6cdf250c47e469a85e923a0ab7f15b3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:14.864598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6cdf250c47e469a85e923a0ab7f15b3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:14.881422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6cdf250c47e469a85e923a0ab7f15b3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:14.890453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1134/2279] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:24:14 2022] DELETE /identity/v3/users/b6cdf250c47e469a85e923a0ab7f15b3 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:24:14.894749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb185e9-c6ee-45a4-9cbe-475c89c3e119 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:14.895229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb185e9-c6ee-45a4-9cbe-475c89c3e119 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:14.895480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb185e9-c6ee-45a4-9cbe-475c89c3e119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:14.919732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0fb185e9-c6ee-45a4-9cbe-475c89c3e119 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:14.921287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0fb185e9-c6ee-45a4-9cbe-475c89c3e119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:14.992272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1146/2280] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:15.005186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:15.006825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:15.040393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:15.070688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cdfc07-8cdb-421c-825d-69c665f518ae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:15.074625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:15.075356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:15.075668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:15.076152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:15.085586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:15.087057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:15.099756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1135/2281] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:24:15.506844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb185e9-c6ee-45a4-9cbe-475c89c3e119 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:15.508296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0fb185e9-c6ee-45a4-9cbe-475c89c3e119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:15.519981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb185e9-c6ee-45a4-9cbe-475c89c3e119 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:15.524555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:15.525969 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:15.526267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:15.526577 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] PATH_INFO: `/v3/projects/d0643dd7b0eb44938b350c31235b0e0c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:15.527817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0643dd7b0eb44938b350c31235b0e0c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:15.529834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:15.563883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:15.653565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0643dd7b0eb44938b350c31235b0e0c', 'request_id': 'req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:15.727902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0643dd7b0eb44938b350c31235b0e0c', 'request_id': 'req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:15.734159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0643dd7b0eb44938b350c31235b0e0c', 'request_id': 'req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:15.746758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] The token cache is being invalidate because project d0643dd7b0eb44938b350c31235b0e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:24:15.747027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:15.747965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:15.749376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1147/2282] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:24:15 2022] DELETE /identity/v3/projects/d0643dd7b0eb44938b350c31235b0e0c => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:24:17.611132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:17.612822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:17.629547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92069d7-afd6-468a-ab0d-70b656843ddc service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:17.688365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:17.688991 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:17.689263 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:17.689530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] PATH_INFO: `/v3/users/75c27b297062483598cf8ff5185e2f2a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:17.690683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c27b297062483598cf8ff5185e2f2a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:17.692676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:17.873483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c27b297062483598cf8ff5185e2f2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:17.946613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c27b297062483598cf8ff5185e2f2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:17.947077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c27b297062483598cf8ff5185e2f2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:17.965962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c27b297062483598cf8ff5185e2f2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:17.979361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c401a6da-caad-41c9-ae27-3237d2d1afd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c27b297062483598cf8ff5185e2f2a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:17.985998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1136/2283] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:24:17 2022] DELETE /identity/v3/users/75c27b297062483598cf8ff5185e2f2a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:24:18.245622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:18.247393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:18.265474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0baaa6b-1f1a-4be7-99ae-78e9d36a271b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:18.329494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:18.330126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:18.330428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:18.330700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] PATH_INFO: `/v3/projects/44a4181be7e6404487bd7eddca1f6d8c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:18.331804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a4181be7e6404487bd7eddca1f6d8c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:18.333707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:18.462198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a4181be7e6404487bd7eddca1f6d8c', 'request_id': 'req-d47e9508-31df-4a64-b162-0fa96ca3c6ca'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:18.532491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a4181be7e6404487bd7eddca1f6d8c', 'request_id': 'req-d47e9508-31df-4a64-b162-0fa96ca3c6ca'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:18.538519 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a4181be7e6404487bd7eddca1f6d8c', 'request_id': 'req-d47e9508-31df-4a64-b162-0fa96ca3c6ca'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:18.555736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] The token cache is being invalidate because project 44a4181be7e6404487bd7eddca1f6d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:24:18.556028 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:18.557325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d47e9508-31df-4a64-b162-0fa96ca3c6ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:18.558850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1148/2284] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:24:18 2022] DELETE /identity/v3/projects/44a4181be7e6404487bd7eddca1f6d8c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:24:18.571340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d313fc-082d-4c51-bcd6-94e1ad5f08ba None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:18.575599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d313fc-082d-4c51-bcd6-94e1ad5f08ba None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:18.575599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d313fc-082d-4c51-bcd6-94e1ad5f08ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:18.591838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-59d313fc-082d-4c51-bcd6-94e1ad5f08ba None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:18.593026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-59d313fc-082d-4c51-bcd6-94e1ad5f08ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:18.634669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:18.652751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1137/2285] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:18.660913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221030e-d42f-4fd8-9764-ef05a92308a9 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:18.661308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221030e-d42f-4fd8-9764-ef05a92308a9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:18.661704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221030e-d42f-4fd8-9764-ef05a92308a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:18.684499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6221030e-d42f-4fd8-9764-ef05a92308a9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:18.685863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6221030e-d42f-4fd8-9764-ef05a92308a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:18.694067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1149/2286] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:18.698873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d313fc-082d-4c51-bcd6-94e1ad5f08ba None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:18.708392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d313fc-082d-4c51-bcd6-94e1ad5f08ba None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:18.713068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:18.713666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:18.713941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:18.714207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:18.714915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:24:18.715207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:18.716776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:18.731915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1138/2287] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:24:18.733565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221030e-d42f-4fd8-9764-ef05a92308a9 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:18.742836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221030e-d42f-4fd8-9764-ef05a92308a9 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:18.748046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc287c83-58ef-448e-b446-f5f79fa5f3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:18.748640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc287c83-58ef-448e-b446-f5f79fa5f3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:18.748908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc287c83-58ef-448e-b446-f5f79fa5f3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:18.749171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc287c83-58ef-448e-b446-f5f79fa5f3a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:18.749801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc287c83-58ef-448e-b446-f5f79fa5f3a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:18.751707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc287c83-58ef-448e-b446-f5f79fa5f3a8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:18.877211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1150/2288] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:24:18 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:24:18.881428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:18.893140 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5e20e-a5e8-47ee-b726-430551eb68fd admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:18.957185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dbee7-34ca-4ac6-b7b2-dc74f7e0bba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:18.957820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dbee7-34ca-4ac6-b7b2-dc74f7e0bba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:18.958100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dbee7-34ca-4ac6-b7b2-dc74f7e0bba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:18.958415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723dbee7-34ca-4ac6-b7b2-dc74f7e0bba9 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:18.959048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dbee7-34ca-4ac6-b7b2-dc74f7e0bba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:18.961012 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723dbee7-34ca-4ac6-b7b2-dc74f7e0bba9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:19.060063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1139/2289] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:24:18 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:24:19.064058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc287c83-58ef-448e-b446-f5f79fa5f3a8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:19.071953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:19.077686 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc287c83-58ef-448e-b446-f5f79fa5f3a8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:19.082307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abebed-99e9-4297-91a9-ad17c448a150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:19.082934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abebed-99e9-4297-91a9-ad17c448a150 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:19.083212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abebed-99e9-4297-91a9-ad17c448a150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:19.095674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abebed-99e9-4297-91a9-ad17c448a150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:19.095674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abebed-99e9-4297-91a9-ad17c448a150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:19.095674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abebed-99e9-4297-91a9-ad17c448a150 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:19.096332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1151/2290] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:24:19.100227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dbee7-34ca-4ac6-b7b2-dc74f7e0bba9 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:19.110899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723dbee7-34ca-4ac6-b7b2-dc74f7e0bba9 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:19.115601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4440eb-46f4-4908-a760-05563df2966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:19.116188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4440eb-46f4-4908-a760-05563df2966f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:19.116465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4440eb-46f4-4908-a760-05563df2966f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:19.116735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4440eb-46f4-4908-a760-05563df2966f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:19.117368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4440eb-46f4-4908-a760-05563df2966f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:19.118977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4440eb-46f4-4908-a760-05563df2966f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:19.129001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1140/2291] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:24:19.133126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abebed-99e9-4297-91a9-ad17c448a150 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:19.147171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abebed-99e9-4297-91a9-ad17c448a150 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:19.153058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:19.153704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:19.153975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:19.154240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] PATH_INFO: `/v3/projects/7b5a865e6ff54be8a063f7d5d1aa1b2e/users/2bd794a64f9f4736b10ae80d896e0937/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:19.170042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5a865e6ff54be8a063f7d5d1aa1b2e, user_id=2bd794a64f9f4736b10ae80d896e0937, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:19.172689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:19.248344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1152/2292] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:24:19 2022] PUT /identity/v3/projects/7b5a865e6ff54be8a063f7d5d1aa1b2e/users/2bd794a64f9f4736b10ae80d896e0937/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:24:19.258631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0c34e-24a5-48d3-a20d-5c322772880f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:19.258919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0c34e-24a5-48d3-a20d-5c322772880f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:19.259221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc0c34e-24a5-48d3-a20d-5c322772880f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:19.290299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fbc0c34e-24a5-48d3-a20d-5c322772880f None None] MFA Rules not processed for user `2bd794a64f9f4736b10ae80d896e0937`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:19.292346 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fbc0c34e-24a5-48d3-a20d-5c322772880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:19.357175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1141/2293] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:24:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:19.369354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:19.371116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:19.428295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e95e4-a97a-47bc-b4b3-bfc7cec75288 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:19.432215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1842c116caa4a5d9d75c8605eeec0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:19.432660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:19.432940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:19.433213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:19.441607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:19.443054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:19.451793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1153/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:24:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:24:21.108463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5426b-61c3-4f39-87fd-638cb146aac5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:21.108961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5426b-61c3-4f39-87fd-638cb146aac5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:21.109054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5426b-61c3-4f39-87fd-638cb146aac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:21.130432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4bf5426b-61c3-4f39-87fd-638cb146aac5 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:21.131647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4bf5426b-61c3-4f39-87fd-638cb146aac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:21.236571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:21.254288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1142/2295] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:24:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:21.266998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:21.268695 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:21.346525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43e3bb-2078-4d7d-9ddc-0e9c171c8d82 service glance] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:21.352000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e823437-b838-4d5d-b8fd-cdb4086c3208 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:21.352600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e823437-b838-4d5d-b8fd-cdb4086c3208 service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:21.352876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e823437-b838-4d5d-b8fd-cdb4086c3208 service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:21.353409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e823437-b838-4d5d-b8fd-cdb4086c3208 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:21.363015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e823437-b838-4d5d-b8fd-cdb4086c3208 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:21.364664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e823437-b838-4d5d-b8fd-cdb4086c3208 service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:21.373918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1154/2296] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:24:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:24:31.316500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=6 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:31.369871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=71,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:40.524761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f25dad-12ae-4141-b888-0495dc11af90 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:40.525164 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f25dad-12ae-4141-b888-0495dc11af90 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:40.525375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f25dad-12ae-4141-b888-0495dc11af90 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:40.526715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1143/2297] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:24:40.533207 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43581e-ed55-4df7-b22f-616c9fed0afb None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:40.533694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43581e-ed55-4df7-b22f-616c9fed0afb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:40.533905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43581e-ed55-4df7-b22f-616c9fed0afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:40.555428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-fe43581e-ed55-4df7-b22f-616c9fed0afb None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:40.556718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fe43581e-ed55-4df7-b22f-616c9fed0afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:40.641688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1155/2298] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:24:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:40.653278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f25dad-12ae-4141-b888-0495dc11af90 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:40.666271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f25dad-12ae-4141-b888-0495dc11af90 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:40.670401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d827177-0249-4d08-beb1-d0fe5029e843 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1842c116caa4a5d9d75c8605eeec0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:40.671082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d827177-0249-4d08-beb1-d0fe5029e843 service glance] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:40.671426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d827177-0249-4d08-beb1-d0fe5029e843 service glance] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:40.671698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d827177-0249-4d08-beb1-d0fe5029e843 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:40.682223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d827177-0249-4d08-beb1-d0fe5029e843 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:40.683823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d827177-0249-4d08-beb1-d0fe5029e843 service glance] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:40.695836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1144/2299] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:24:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:24:40.924005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0930-3c60-4688-860c-a167ab903794 None None] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:40.924395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0930-3c60-4688-860c-a167ab903794 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:40.924677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed0930-3c60-4688-860c-a167ab903794 None None] PATH_INFO: `/` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:40.926014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1156/2300] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:24:40.931033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2d3c-2e70-45c0-8d72-1b9cf84b3590 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:40.931314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2d3c-2e70-45c0-8d72-1b9cf84b3590 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:40.931580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d2d3c-2e70-45c0-8d72-1b9cf84b3590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:40.959604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a24d2d3c-2e70-45c0-8d72-1b9cf84b3590 None None] MFA Rules not processed for user `a9da2d7ef80f4622b58c9041b65df992`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:40.960904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a24d2d3c-2e70-45c0-8d72-1b9cf84b3590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:41.019747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1145/2301] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:24:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:41.032286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed0930-3c60-4688-860c-a167ab903794 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:41.033984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-42ed0930-3c60-4688-860c-a167ab903794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:41.047773 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed0930-3c60-4688-860c-a167ab903794 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:41.051399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde26d1-9e83-4ac1-a834-10b9828a51c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:41.051987 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde26d1-9e83-4ac1-a834-10b9828a51c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:41.052259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde26d1-9e83-4ac1-a834-10b9828a51c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:41.052525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde26d1-9e83-4ac1-a834-10b9828a51c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:41.059718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde26d1-9e83-4ac1-a834-10b9828a51c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:41.061527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde26d1-9e83-4ac1-a834-10b9828a51c1 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:41.069337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1157/2302] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:24:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:24:41.152158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528945cc-9fcc-4871-bf4c-9f1684a8bc86 None None] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:41.152459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528945cc-9fcc-4871-bf4c-9f1684a8bc86 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:41.152714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528945cc-9fcc-4871-bf4c-9f1684a8bc86 None None] PATH_INFO: `/` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:41.154895 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1146/2303] 10.222.0.20 () {54 vars in 921 bytes} [Fri Sep 9 23:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 09 23:24:41.160684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed33873-50ef-4194-9ad8-c93d7103bc09 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:41.160967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed33873-50ef-4194-9ad8-c93d7103bc09 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:41.161235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed33873-50ef-4194-9ad8-c93d7103bc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:41.181143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-fed33873-50ef-4194-9ad8-c93d7103bc09 None None] MFA Rules not processed for user `c3bec982c8474c349ca4901039844d26`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:41.182264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fed33873-50ef-4194-9ad8-c93d7103bc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:41.191461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1158/2304] 10.222.0.20 () {58 vars in 1018 bytes} [Fri Sep 9 23:24:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:41.201657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528945cc-9fcc-4871-bf4c-9f1684a8bc86 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:41.212598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528945cc-9fcc-4871-bf4c-9f1684a8bc86 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:41.216249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2bce9-a322-4270-a6be-e51a3a930f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:41.217385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2bce9-a322-4270-a6be-e51a3a930f01 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:41.217660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2bce9-a322-4270-a6be-e51a3a930f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:41.217929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2bce9-a322-4270-a6be-e51a3a930f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:41.228497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2bce9-a322-4270-a6be-e51a3a930f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:41.229914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2bce9-a322-4270-a6be-e51a3a930f01 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:41.241889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1147/2305] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:24:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:24:51.176415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=40 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:51.236908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=35 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:57.723846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed33873-50ef-4194-9ad8-c93d7103bc09 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:57.725772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fed33873-50ef-4194-9ad8-c93d7103bc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:57.808075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed33873-50ef-4194-9ad8-c93d7103bc09 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:57.813748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:57.814417 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:57.814696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:57.814964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] PATH_INFO: `/v3/users/72467431e40c491cad192b06be494175` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:57.816079 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=72467431e40c491cad192b06be494175)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:57.819243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:58.026677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72467431e40c491cad192b06be494175'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.027337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72467431e40c491cad192b06be494175'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.120360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72467431e40c491cad192b06be494175'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.133175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72467431e40c491cad192b06be494175'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.141383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3630a54c-445a-463e-bd25-5f162996ee2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72467431e40c491cad192b06be494175'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.147402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1159/2306] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:24:57 2022] DELETE /identity/v3/users/72467431e40c491cad192b06be494175 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:24:58.471871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2bce9-a322-4270-a6be-e51a3a930f01 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:58.486279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2bce9-a322-4270-a6be-e51a3a930f01 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:58.547559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:58.548186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:58.548459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:58.548727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] PATH_INFO: `/v3/projects/c31116d92493460793d41ba2a6a166e0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:58.549934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31116d92493460793d41ba2a6a166e0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:58.551898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:58.629972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31116d92493460793d41ba2a6a166e0', 'request_id': 'req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.669830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31116d92493460793d41ba2a6a166e0', 'request_id': 'req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.675642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31116d92493460793d41ba2a6a166e0', 'request_id': 'req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.686909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] The token cache is being invalidate because project c31116d92493460793d41ba2a6a166e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:24:58.687195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.687818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-326dcf2d-9aa8-43d8-ad02-11bdf85b5cc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:24:58.689284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1148/2307] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:24:58 2022] DELETE /identity/v3/projects/c31116d92493460793d41ba2a6a166e0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:24:58.694312 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171803ec-1b96-42e4-bc03-4ac358520480 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:58.694614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171803ec-1b96-42e4-bc03-4ac358520480 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:58.694883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171803ec-1b96-42e4-bc03-4ac358520480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:58.714570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-171803ec-1b96-42e4-bc03-4ac358520480 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:58.715763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-171803ec-1b96-42e4-bc03-4ac358520480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:58.774425 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1160/2308] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:24:58.781980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f1a2-8522-4559-a116-23d83ce9ba5e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:58.782291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f1a2-8522-4559-a116-23d83ce9ba5e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:58.782590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428f1a2-8522-4559-a116-23d83ce9ba5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:58.802665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b428f1a2-8522-4559-a116-23d83ce9ba5e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:58.803875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b428f1a2-8522-4559-a116-23d83ce9ba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:58.812308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1149/2309] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:58.816353 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171803ec-1b96-42e4-bc03-4ac358520480 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:58.826019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171803ec-1b96-42e4-bc03-4ac358520480 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:58.830612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:58.831202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:58.831476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:58.831745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:58.832383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:24:58.832708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:58.834278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:58.845730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1161/2310] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:24:58.849632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f1a2-8522-4559-a116-23d83ce9ba5e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:58.859699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428f1a2-8522-4559-a116-23d83ce9ba5e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:58.864536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff69a76-a169-4d4b-ace7-78ff964e6da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:58.865126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff69a76-a169-4d4b-ace7-78ff964e6da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:58.865396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff69a76-a169-4d4b-ace7-78ff964e6da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:58.865659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff69a76-a169-4d4b-ace7-78ff964e6da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:58.866286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff69a76-a169-4d4b-ace7-78ff964e6da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:58.868230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff69a76-a169-4d4b-ace7-78ff964e6da5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:58.940548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1150/2311] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:24:58 2022] POST /identity/v3/projects => generated 389 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:24:58.945121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:58.956732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fb13f-177d-4b86-9f3b-cb61ae3a8b7d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:58.960513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:59.034544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5e85-8f87-412f-96f1-080c76a5d8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:59.035156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5e85-8f87-412f-96f1-080c76a5d8ed admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:59.035440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5e85-8f87-412f-96f1-080c76a5d8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:59.035703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5e85-8f87-412f-96f1-080c76a5d8ed admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:59.036338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc5e85-8f87-412f-96f1-080c76a5d8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:59.038216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc5e85-8f87-412f-96f1-080c76a5d8ed admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:59.162441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1162/2312] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:24:58 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:24:59.166640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff69a76-a169-4d4b-ace7-78ff964e6da5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:59.177098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff69a76-a169-4d4b-ace7-78ff964e6da5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:59.181675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31a33a-df70-4e11-808d-2cf2d5e43d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:59.182260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a33a-df70-4e11-808d-2cf2d5e43d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:59.182566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a33a-df70-4e11-808d-2cf2d5e43d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:59.182837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a33a-df70-4e11-808d-2cf2d5e43d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:59.183469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31a33a-df70-4e11-808d-2cf2d5e43d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:59.185141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31a33a-df70-4e11-808d-2cf2d5e43d30 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:59.200081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1151/2313] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:24:59.201286 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5e85-8f87-412f-96f1-080c76a5d8ed admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:59.212004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5e85-8f87-412f-96f1-080c76a5d8ed admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:59.216752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bf0ba-760a-45de-9a2b-6dfac0e789d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:59.217437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bf0ba-760a-45de-9a2b-6dfac0e789d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:59.217792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bf0ba-760a-45de-9a2b-6dfac0e789d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:59.221384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bf0ba-760a-45de-9a2b-6dfac0e789d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:59.221384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bf0ba-760a-45de-9a2b-6dfac0e789d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:59.221384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bf0ba-760a-45de-9a2b-6dfac0e789d6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:59.230601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1163/2314] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:24:59.235360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31a33a-df70-4e11-808d-2cf2d5e43d30 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:59.245445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31a33a-df70-4e11-808d-2cf2d5e43d30 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:59.251067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:59.251726 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:59.252011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:59.252282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] PATH_INFO: `/v3/projects/b87bcd0e596f4e7f8d9534da2056f86c/users/73a16e11ca684ae18510885d274d7c4d/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:59.267736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87bcd0e596f4e7f8d9534da2056f86c, user_id=73a16e11ca684ae18510885d274d7c4d, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:59.270407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:59.356116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1152/2315] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:24:59 2022] PUT /identity/v3/projects/b87bcd0e596f4e7f8d9534da2056f86c/users/73a16e11ca684ae18510885d274d7c4d/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:24:59.368170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34575281-a633-4f3f-8f2c-d5eb29263cee None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:59.368465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34575281-a633-4f3f-8f2c-d5eb29263cee None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:59.368779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34575281-a633-4f3f-8f2c-d5eb29263cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:59.400088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-34575281-a633-4f3f-8f2c-d5eb29263cee None None] MFA Rules not processed for user `73a16e11ca684ae18510885d274d7c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:24:59.401300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-34575281-a633-4f3f-8f2c-d5eb29263cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:59.481943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1164/2316] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:24:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:24:59.493785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:24:59.495594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:24:59.528542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:24:59.557822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10c609-9648-4792-9cd9-4c8cf3835113 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:24:59.561777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd32db-f9a7-4891-8c4d-e4bf36bd503d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:24:59.562418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd32db-f9a7-4891-8c4d-e4bf36bd503d service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:24:59.562691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd32db-f9a7-4891-8c4d-e4bf36bd503d service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:24:59.562954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd32db-f9a7-4891-8c4d-e4bf36bd503d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:24:59.571363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd32db-f9a7-4891-8c4d-e4bf36bd503d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:24:59.572776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd32db-f9a7-4891-8c4d-e4bf36bd503d service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:24:59.582569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1153/2317] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:24:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:01.581036 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34575281-a633-4f3f-8f2c-d5eb29263cee None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:01.582589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-34575281-a633-4f3f-8f2c-d5eb29263cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:01.654248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34575281-a633-4f3f-8f2c-d5eb29263cee None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:01.660968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:01.661738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:01.661900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:01.662171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] PATH_INFO: `/v3/users/2bd794a64f9f4736b10ae80d896e0937` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:01.663328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bd794a64f9f4736b10ae80d896e0937)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:01.665229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:01.687377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:01.867365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bd794a64f9f4736b10ae80d896e0937'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:01.868013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bd794a64f9f4736b10ae80d896e0937'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:01.955626 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bd794a64f9f4736b10ae80d896e0937'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:01.970514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bd794a64f9f4736b10ae80d896e0937'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:01.977631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bd794a64f9f4736b10ae80d896e0937'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:01.984002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1165/2318] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:25:01 2022] DELETE /identity/v3/users/2bd794a64f9f4736b10ae80d896e0937 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:01.988544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dacf99-d42c-4d16-998b-fb0ba4b5d9be None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:01.995572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dacf99-d42c-4d16-998b-fb0ba4b5d9be None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:01.995572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dacf99-d42c-4d16-998b-fb0ba4b5d9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:02.009187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-96dacf99-d42c-4d16-998b-fb0ba4b5d9be None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:02.010404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-96dacf99-d42c-4d16-998b-fb0ba4b5d9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:02.067171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1154/2319] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:02.077347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:02.078813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:02.135390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fbe72-ac92-4a5d-8452-71a0b18b577b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:02.139118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd548f-da29-4dab-9ac7-89a3e9ed8413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:02.139742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd548f-da29-4dab-9ac7-89a3e9ed8413 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:02.140019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd548f-da29-4dab-9ac7-89a3e9ed8413 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:02.140290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd548f-da29-4dab-9ac7-89a3e9ed8413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:02.149333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd548f-da29-4dab-9ac7-89a3e9ed8413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:02.150834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cd548f-da29-4dab-9ac7-89a3e9ed8413 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:02.160877 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1166/2320] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:02.576076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dacf99-d42c-4d16-998b-fb0ba4b5d9be None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:02.578086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-96dacf99-d42c-4d16-998b-fb0ba4b5d9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:02.600018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dacf99-d42c-4d16-998b-fb0ba4b5d9be None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:02.604829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:02.605517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:02.605797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:02.606064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] PATH_INFO: `/v3/projects/7b5a865e6ff54be8a063f7d5d1aa1b2e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:02.607215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5a865e6ff54be8a063f7d5d1aa1b2e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:02.610101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:02.716104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5a865e6ff54be8a063f7d5d1aa1b2e', 'request_id': 'req-d20bdd6c-9bdf-443b-9624-e2692e90e78e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:02.783072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5a865e6ff54be8a063f7d5d1aa1b2e', 'request_id': 'req-d20bdd6c-9bdf-443b-9624-e2692e90e78e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:02.789037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5a865e6ff54be8a063f7d5d1aa1b2e', 'request_id': 'req-d20bdd6c-9bdf-443b-9624-e2692e90e78e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:02.801539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] The token cache is being invalidate because project 7b5a865e6ff54be8a063f7d5d1aa1b2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:25:02.801822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:02.802518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d20bdd6c-9bdf-443b-9624-e2692e90e78e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:02.803997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1155/2321] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:25:02 2022] DELETE /identity/v3/projects/7b5a865e6ff54be8a063f7d5d1aa1b2e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:02.809274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b84d-ee54-401c-b250-987ff28d54a9 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:02.809737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b84d-ee54-401c-b250-987ff28d54a9 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:02.809832 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43b84d-ee54-401c-b250-987ff28d54a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:02.829090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5b43b84d-ee54-401c-b250-987ff28d54a9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:02.830261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5b43b84d-ee54-401c-b250-987ff28d54a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:02.887952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1167/2322] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:02.952099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c929c78-5994-490d-ba2c-857a5836071b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:02.952421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c929c78-5994-490d-ba2c-857a5836071b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:02.952694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c929c78-5994-490d-ba2c-857a5836071b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:02.972876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0c929c78-5994-490d-ba2c-857a5836071b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:02.974088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0c929c78-5994-490d-ba2c-857a5836071b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:02.982054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1156/2323] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:02.987143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b84d-ee54-401c-b250-987ff28d54a9 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:02.997337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43b84d-ee54-401c-b250-987ff28d54a9 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:03.002380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:03.002986 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:03.003260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:03.003529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:03.004175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:25:03.004478 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:03.006067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:03.017714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1168/2324] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:25:03.022082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c929c78-5994-490d-ba2c-857a5836071b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:03.031939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c929c78-5994-490d-ba2c-857a5836071b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:03.036642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abab5dd-47eb-4065-863b-4097c4863ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:03.037239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abab5dd-47eb-4065-863b-4097c4863ede admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:03.037516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abab5dd-47eb-4065-863b-4097c4863ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:03.037785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abab5dd-47eb-4065-863b-4097c4863ede admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:03.038453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abab5dd-47eb-4065-863b-4097c4863ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:03.040352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abab5dd-47eb-4065-863b-4097c4863ede admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:03.123406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1157/2325] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:25:03 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:25:03.127806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:03.138498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba829d1-c5a7-4fb5-ad8c-cda37e83ca75 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:03.162893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:03.198925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecba6ee-9ae1-48c2-a35b-9ef83ff1fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:03.199521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecba6ee-9ae1-48c2-a35b-9ef83ff1fb5d admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:03.199794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecba6ee-9ae1-48c2-a35b-9ef83ff1fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:03.200057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecba6ee-9ae1-48c2-a35b-9ef83ff1fb5d admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:03.200680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecba6ee-9ae1-48c2-a35b-9ef83ff1fb5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:03.203178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecba6ee-9ae1-48c2-a35b-9ef83ff1fb5d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:03.315680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1169/2326] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:25:03 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:25:03.319838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abab5dd-47eb-4065-863b-4097c4863ede admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:03.331776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abab5dd-47eb-4065-863b-4097c4863ede admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:03.336294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93b564-994d-47da-bfb1-fd0d380cf876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:03.336887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93b564-994d-47da-bfb1-fd0d380cf876 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:03.337159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93b564-994d-47da-bfb1-fd0d380cf876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:03.337424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93b564-994d-47da-bfb1-fd0d380cf876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:03.338053 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93b564-994d-47da-bfb1-fd0d380cf876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:03.339792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93b564-994d-47da-bfb1-fd0d380cf876 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:03.350076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1158/2327] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:03.354137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecba6ee-9ae1-48c2-a35b-9ef83ff1fb5d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:03.364766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecba6ee-9ae1-48c2-a35b-9ef83ff1fb5d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:03.369307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4e1ce-aeb0-48c7-9223-8b62c8e8dfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:03.369888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4e1ce-aeb0-48c7-9223-8b62c8e8dfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:03.370160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4e1ce-aeb0-48c7-9223-8b62c8e8dfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:03.370463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4e1ce-aeb0-48c7-9223-8b62c8e8dfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:03.371128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4e1ce-aeb0-48c7-9223-8b62c8e8dfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:03.372684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4e1ce-aeb0-48c7-9223-8b62c8e8dfc1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:03.382919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1170/2328] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:03.386887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93b564-994d-47da-bfb1-fd0d380cf876 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:03.397613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93b564-994d-47da-bfb1-fd0d380cf876 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:03.402130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:03.402807 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:03.403081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:03.403347 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] PATH_INFO: `/v3/projects/d4694a14825f4600a0336e150452a9cf/users/4398271cfc094e81b7a120bdeeb6dafe/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:03.414957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:03.419143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4694a14825f4600a0336e150452a9cf, user_id=4398271cfc094e81b7a120bdeeb6dafe, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:03.422631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:03.533273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1159/2329] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:25:03 2022] PUT /identity/v3/projects/d4694a14825f4600a0336e150452a9cf/users/4398271cfc094e81b7a120bdeeb6dafe/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:25:03.543437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dfb9f-df27-4e18-8f19-74236e42bc91 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:03.543721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dfb9f-df27-4e18-8f19-74236e42bc91 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:03.543991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dfb9f-df27-4e18-8f19-74236e42bc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:03.571219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-724dfb9f-df27-4e18-8f19-74236e42bc91 None None] MFA Rules not processed for user `4398271cfc094e81b7a120bdeeb6dafe`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:03.572408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-724dfb9f-df27-4e18-8f19-74236e42bc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:03.640596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1171/2330] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:03.651311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:03.653838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:03.723729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe826e-ddc9-4bc5-94e8-5e9222fcdf27 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:03.728209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ab287-7b80-4a30-a8a9-2a86598ddc9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:03.729075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab287-7b80-4a30-a8a9-2a86598ddc9e service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:03.729416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab287-7b80-4a30-a8a9-2a86598ddc9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:03.729700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab287-7b80-4a30-a8a9-2a86598ddc9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:03.739402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ab287-7b80-4a30-a8a9-2a86598ddc9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:03.740833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ab287-7b80-4a30-a8a9-2a86598ddc9e service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:03.751188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1160/2331] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:05.462302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906555e0-95ac-49f0-a8b8-7ec0cec3ea8a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:05.462879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906555e0-95ac-49f0-a8b8-7ec0cec3ea8a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:05.463344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906555e0-95ac-49f0-a8b8-7ec0cec3ea8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:05.482285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-906555e0-95ac-49f0-a8b8-7ec0cec3ea8a None None] MFA Rules not processed for user `4398271cfc094e81b7a120bdeeb6dafe`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:05.485051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-906555e0-95ac-49f0-a8b8-7ec0cec3ea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:05.492153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1172/2332] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:13.745817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:15.477465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:17.293733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96c48f-6367-4d17-b64e-11db4ce7e865 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:17.294257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96c48f-6367-4d17-b64e-11db4ce7e865 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:17.294257 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96c48f-6367-4d17-b64e-11db4ce7e865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:17.314625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8b96c48f-6367-4d17-b64e-11db4ce7e865 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:17.315839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8b96c48f-6367-4d17-b64e-11db4ce7e865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:17.383073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1161/2333] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:25:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:17.408429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906555e0-95ac-49f0-a8b8-7ec0cec3ea8a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:17.409999 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-906555e0-95ac-49f0-a8b8-7ec0cec3ea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:17.501705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906555e0-95ac-49f0-a8b8-7ec0cec3ea8a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:17.506896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e05e71-0ce1-4264-a059-59fd743e4478 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:17.507974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05e71-0ce1-4264-a059-59fd743e4478 service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:17.508264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05e71-0ce1-4264-a059-59fd743e4478 service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:17.508525 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e05e71-0ce1-4264-a059-59fd743e4478 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:17.518730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e05e71-0ce1-4264-a059-59fd743e4478 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:17.520352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e05e71-0ce1-4264-a059-59fd743e4478 service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:17.530089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1173/2334] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:25:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:25:21.329327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96c48f-6367-4d17-b64e-11db4ce7e865 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:21.331358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8b96c48f-6367-4d17-b64e-11db4ce7e865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:21.441752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96c48f-6367-4d17-b64e-11db4ce7e865 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:21.446610 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:21.447289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:21.447623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:21.448113 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] PATH_INFO: `/v3/users/73a16e11ca684ae18510885d274d7c4d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:21.449290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a16e11ca684ae18510885d274d7c4d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:21.451309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:21.656121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a16e11ca684ae18510885d274d7c4d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:21.722366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a16e11ca684ae18510885d274d7c4d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:21.723009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a16e11ca684ae18510885d274d7c4d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:21.732124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a16e11ca684ae18510885d274d7c4d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:21.757002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a16e11ca684ae18510885d274d7c4d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:21.763776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1162/2335] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:25:21 2022] DELETE /identity/v3/users/73a16e11ca684ae18510885d274d7c4d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:21.769936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131759a6-2426-486a-ad7c-57604c191533 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:21.770723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131759a6-2426-486a-ad7c-57604c191533 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:21.772359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131759a6-2426-486a-ad7c-57604c191533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:21.803314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-131759a6-2426-486a-ad7c-57604c191533 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:21.804937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-131759a6-2426-486a-ad7c-57604c191533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:21.884806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1174/2336] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:21.900227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:21.901721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:21.937247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:21.975694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe32fc-974a-49f4-8f07-6aa776d97f68 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:21.979870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c5bbd-d4bd-4e90-a1ef-11b475a5c810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:21.980479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c5bbd-d4bd-4e90-a1ef-11b475a5c810 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:21.984189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c5bbd-d4bd-4e90-a1ef-11b475a5c810 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:21.984189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c5bbd-d4bd-4e90-a1ef-11b475a5c810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:21.993097 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c5bbd-d4bd-4e90-a1ef-11b475a5c810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:21.994633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50c5bbd-d4bd-4e90-a1ef-11b475a5c810 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:22.006521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1163/2337] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:22.476118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131759a6-2426-486a-ad7c-57604c191533 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:22.477799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-131759a6-2426-486a-ad7c-57604c191533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:22.506769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131759a6-2426-486a-ad7c-57604c191533 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:22.521579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:22.522396 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:22.522775 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:22.523142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] PATH_INFO: `/v3/projects/b87bcd0e596f4e7f8d9534da2056f86c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:22.531574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b87bcd0e596f4e7f8d9534da2056f86c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:22.535048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:22.679710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b87bcd0e596f4e7f8d9534da2056f86c', 'request_id': 'req-6b42b882-d82d-4204-b048-6448d610f674'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:22.761297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b87bcd0e596f4e7f8d9534da2056f86c', 'request_id': 'req-6b42b882-d82d-4204-b048-6448d610f674'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:22.770162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b87bcd0e596f4e7f8d9534da2056f86c', 'request_id': 'req-6b42b882-d82d-4204-b048-6448d610f674'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:22.782727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] The token cache is being invalidate because project b87bcd0e596f4e7f8d9534da2056f86c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:25:22.783090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:22.783805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6b42b882-d82d-4204-b048-6448d610f674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:22.786475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1175/2338] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:25:22 2022] DELETE /identity/v3/projects/b87bcd0e596f4e7f8d9534da2056f86c => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:22.791172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cc2e6-0ecd-477f-ae14-52d157bb42d9 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:22.794279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cc2e6-0ecd-477f-ae14-52d157bb42d9 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:22.794279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cc2e6-0ecd-477f-ae14-52d157bb42d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:22.821566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7f0cc2e6-0ecd-477f-ae14-52d157bb42d9 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:22.823302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7f0cc2e6-0ecd-477f-ae14-52d157bb42d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:22.927434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1164/2339] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:22.935090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c23987-0160-4563-8903-42c7b3c6522b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:22.935284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c23987-0160-4563-8903-42c7b3c6522b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:22.935561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c23987-0160-4563-8903-42c7b3c6522b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:22.989131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-82c23987-0160-4563-8903-42c7b3c6522b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:23.001008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-82c23987-0160-4563-8903-42c7b3c6522b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:23.016423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1176/2340] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:25:23.017844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cc2e6-0ecd-477f-ae14-52d157bb42d9 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:23.028725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cc2e6-0ecd-477f-ae14-52d157bb42d9 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:23.037747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:23.037747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:23.037747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:23.037747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:23.038359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:25:23.038359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:23.038746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:23.055515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1165/2341] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:25:23.059256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c23987-0160-4563-8903-42c7b3c6522b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:23.074534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c23987-0160-4563-8903-42c7b3c6522b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:23.080295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d673b5a-5fde-4d8d-aa67-0400b47dbf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:23.080902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d673b5a-5fde-4d8d-aa67-0400b47dbf5e admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:23.081172 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d673b5a-5fde-4d8d-aa67-0400b47dbf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:23.081434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d673b5a-5fde-4d8d-aa67-0400b47dbf5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:23.082061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d673b5a-5fde-4d8d-aa67-0400b47dbf5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:23.083988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d673b5a-5fde-4d8d-aa67-0400b47dbf5e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:23.088683 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:23.220870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1177/2342] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:25:23 2022] POST /identity/v3/projects => generated 399 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:25:23.224758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:23.236984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c988f6c6-3b01-40bf-b37a-dd8cb5b46fb7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:23.302378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c1bc9-2ef6-4cf6-be22-195389c6df57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:23.303161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c1bc9-2ef6-4cf6-be22-195389c6df57 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:23.303538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c1bc9-2ef6-4cf6-be22-195389c6df57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:23.303899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c1bc9-2ef6-4cf6-be22-195389c6df57 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:23.304632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c1bc9-2ef6-4cf6-be22-195389c6df57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:23.306934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c1bc9-2ef6-4cf6-be22-195389c6df57 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:23.384649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1166/2343] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:25:23 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:25:23.388768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d673b5a-5fde-4d8d-aa67-0400b47dbf5e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:23.403261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d673b5a-5fde-4d8d-aa67-0400b47dbf5e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:23.408799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840e92e-f76c-4d61-a89f-487db24b3bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:23.409410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e92e-f76c-4d61-a89f-487db24b3bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:23.409687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e92e-f76c-4d61-a89f-487db24b3bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:23.410091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840e92e-f76c-4d61-a89f-487db24b3bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:23.414440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840e92e-f76c-4d61-a89f-487db24b3bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:23.414440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840e92e-f76c-4d61-a89f-487db24b3bb0 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:23.431487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1178/2344] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:23.434131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c1bc9-2ef6-4cf6-be22-195389c6df57 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:23.445035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c1bc9-2ef6-4cf6-be22-195389c6df57 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:23.449747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d7aec0-14a6-41c9-bd12-ba2e1ce3586c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:23.450361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7aec0-14a6-41c9-bd12-ba2e1ce3586c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:23.450658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7aec0-14a6-41c9-bd12-ba2e1ce3586c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:23.450921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d7aec0-14a6-41c9-bd12-ba2e1ce3586c admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:23.452133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7aec0-14a6-41c9-bd12-ba2e1ce3586c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:23.455305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d7aec0-14a6-41c9-bd12-ba2e1ce3586c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:23.465088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1167/2345] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:23.470179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840e92e-f76c-4d61-a89f-487db24b3bb0 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:23.483540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840e92e-f76c-4d61-a89f-487db24b3bb0 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:23.489941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:23.490721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:23.491069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:23.491420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] PATH_INFO: `/v3/projects/b944a8906c914b7ab80e00f2422b53d2/users/169bcc2ce381421aa6f00e517e0dc786/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:23.507753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b944a8906c914b7ab80e00f2422b53d2, user_id=169bcc2ce381421aa6f00e517e0dc786, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:23.510726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:23.566849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1179/2346] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:25:23 2022] PUT /identity/v3/projects/b944a8906c914b7ab80e00f2422b53d2/users/169bcc2ce381421aa6f00e517e0dc786/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:23.579101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a1113-f7d6-41e3-bb23-60b7cb467474 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:23.579751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a1113-f7d6-41e3-bb23-60b7cb467474 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:23.580373 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a1113-f7d6-41e3-bb23-60b7cb467474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:23.633755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-588a1113-f7d6-41e3-bb23-60b7cb467474 None None] MFA Rules not processed for user `169bcc2ce381421aa6f00e517e0dc786`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:23.642621 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-588a1113-f7d6-41e3-bb23-60b7cb467474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:23.706721 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:23.777254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1168/2347] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:23.798857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:23.801135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:23.882373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104695e2-767d-4dd6-93b8-dd93167b356c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:23.882373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb3845-19b7-4418-92ce-cb40fc73ef92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:23.882373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb3845-19b7-4418-92ce-cb40fc73ef92 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:23.882373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb3845-19b7-4418-92ce-cb40fc73ef92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:23.882869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb3845-19b7-4418-92ce-cb40fc73ef92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:23.898977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb3845-19b7-4418-92ce-cb40fc73ef92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:23.900475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb3845-19b7-4418-92ce-cb40fc73ef92 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:23.919369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1180/2348] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:33.072818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a1113-f7d6-41e3-bb23-60b7cb467474 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:33.141522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a1113-f7d6-41e3-bb23-60b7cb467474 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:33.146172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:33.146827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:33.147103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:33.147392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] PATH_INFO: `/v3/users/169bcc2ce381421aa6f00e517e0dc786` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:33.148537 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=169bcc2ce381421aa6f00e517e0dc786)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:33.150495 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:33.341055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '169bcc2ce381421aa6f00e517e0dc786'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:33.418107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '169bcc2ce381421aa6f00e517e0dc786'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:33.418823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '169bcc2ce381421aa6f00e517e0dc786'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:33.427615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '169bcc2ce381421aa6f00e517e0dc786'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:33.440208 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '169bcc2ce381421aa6f00e517e0dc786'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:33.446152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1169/2349] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:25:33 2022] DELETE /identity/v3/users/169bcc2ce381421aa6f00e517e0dc786 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:33.450643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db6f7-4a0c-44d7-ae08-d3b3cc9b65fb None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:33.451149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db6f7-4a0c-44d7-ae08-d3b3cc9b65fb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:33.451149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8db6f7-4a0c-44d7-ae08-d3b3cc9b65fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:33.470984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4e8db6f7-4a0c-44d7-ae08-d3b3cc9b65fb None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:33.472165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4e8db6f7-4a0c-44d7-ae08-d3b3cc9b65fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:33.529593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1181/2350] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:33.539536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:33.540931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:33.588577 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:33.602842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd35188-9767-4a01-846b-3e95d9f5e16e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:33.604332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:33.604982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:33.605379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:33.605655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:33.616018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:33.617817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:33.628830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1170/2351] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:33.773355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c697c-497b-4de1-836e-190f10ee4f3c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:33.773638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c697c-497b-4de1-836e-190f10ee4f3c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:33.773955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c697c-497b-4de1-836e-190f10ee4f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:33.802977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e74c697c-497b-4de1-836e-190f10ee4f3c None None] MFA Rules not processed for user `987d2f6bb1ff4f1fa248c3ff410c8119`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:33.804200 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e74c697c-497b-4de1-836e-190f10ee4f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:33.873937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:33.883243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1182/2352] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:34.248738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:34.251040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:34.272199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9685be-5624-4b3e-9733-71f1114f9046 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:34.278141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:34.278917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:34.279143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:34.279492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] PATH_INFO: `/v3/projects/b944a8906c914b7ab80e00f2422b53d2` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:34.280983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b944a8906c914b7ab80e00f2422b53d2)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:34.283133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:34.478277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b944a8906c914b7ab80e00f2422b53d2', 'request_id': 'req-cbff5075-4cc8-4688-ae25-f073ec9ffd20'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:34.537979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b944a8906c914b7ab80e00f2422b53d2', 'request_id': 'req-cbff5075-4cc8-4688-ae25-f073ec9ffd20'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:34.544573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b944a8906c914b7ab80e00f2422b53d2', 'request_id': 'req-cbff5075-4cc8-4688-ae25-f073ec9ffd20'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:34.556997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] The token cache is being invalidate because project b944a8906c914b7ab80e00f2422b53d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:25:34.557290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:34.557940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cbff5075-4cc8-4688-ae25-f073ec9ffd20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:34.559624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1171/2353] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:25:34 2022] DELETE /identity/v3/projects/b944a8906c914b7ab80e00f2422b53d2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:34.565034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c7391-20d5-48fa-98dd-ab918387b900 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:34.565560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c7391-20d5-48fa-98dd-ab918387b900 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:34.565704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c7391-20d5-48fa-98dd-ab918387b900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:34.588363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2d0c7391-20d5-48fa-98dd-ab918387b900 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:34.589884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2d0c7391-20d5-48fa-98dd-ab918387b900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:34.652181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1183/2354] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:34.660707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7979a10-57e2-4fa6-be80-e0c91b344250 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:34.660920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7979a10-57e2-4fa6-be80-e0c91b344250 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:34.661209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7979a10-57e2-4fa6-be80-e0c91b344250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:34.680683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c7979a10-57e2-4fa6-be80-e0c91b344250 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:34.681868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c7979a10-57e2-4fa6-be80-e0c91b344250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:34.690083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1172/2355] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:34.694719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0c7391-20d5-48fa-98dd-ab918387b900 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:34.706762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0c7391-20d5-48fa-98dd-ab918387b900 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:34.711502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:34.712100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:34.712372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:34.712635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:34.714650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:25:34.714976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:34.716561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:34.730724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1184/2356] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:25:34.735994 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7979a10-57e2-4fa6-be80-e0c91b344250 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:34.746492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7979a10-57e2-4fa6-be80-e0c91b344250 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:34.752015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a4f25-6196-4ce0-8299-3c231896e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:34.752690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a4f25-6196-4ce0-8299-3c231896e57c admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:34.752963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a4f25-6196-4ce0-8299-3c231896e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:34.753226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a4f25-6196-4ce0-8299-3c231896e57c admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:34.753849 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a4f25-6196-4ce0-8299-3c231896e57c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:34.755870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a4f25-6196-4ce0-8299-3c231896e57c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:34.847729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1173/2357] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:25:34 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:25:34.851817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:34.864105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe18447-10d6-4933-9580-2ce79115ffcf admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:34.921854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8c38-709f-4280-af4a-a1c8b32baa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:34.922335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8c38-709f-4280-af4a-a1c8b32baa5b admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:34.922638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8c38-709f-4280-af4a-a1c8b32baa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:34.922909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8c38-709f-4280-af4a-a1c8b32baa5b admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:34.923515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e8c38-709f-4280-af4a-a1c8b32baa5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:34.925398 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e8c38-709f-4280-af4a-a1c8b32baa5b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:34.996457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1185/2358] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:25:34 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:25:35.000824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a4f25-6196-4ce0-8299-3c231896e57c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:35.012627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a4f25-6196-4ce0-8299-3c231896e57c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:35.019081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac6281-3098-4c63-9875-57646c143394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:35.019796 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac6281-3098-4c63-9875-57646c143394 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:35.020173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac6281-3098-4c63-9875-57646c143394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:35.020566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac6281-3098-4c63-9875-57646c143394 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:35.021335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac6281-3098-4c63-9875-57646c143394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:35.023362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac6281-3098-4c63-9875-57646c143394 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:35.035618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1174/2359] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:25:35.039017 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8c38-709f-4280-af4a-a1c8b32baa5b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:35.049469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8c38-709f-4280-af4a-a1c8b32baa5b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:35.055720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28351e18-2ba4-4992-aa8f-fab047492fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:35.056328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28351e18-2ba4-4992-aa8f-fab047492fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:35.056606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28351e18-2ba4-4992-aa8f-fab047492fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:35.056881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28351e18-2ba4-4992-aa8f-fab047492fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:35.057514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28351e18-2ba4-4992-aa8f-fab047492fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:35.059163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28351e18-2ba4-4992-aa8f-fab047492fcf admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:35.070648 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1186/2360] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:35.074711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac6281-3098-4c63-9875-57646c143394 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:35.087136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac6281-3098-4c63-9875-57646c143394 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:35.094995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:35.096127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:35.096410 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:35.096677 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] PATH_INFO: `/v3/projects/28d0bb1d968841c2986b5eb164af953b/users/c4129e5f14794a619acdaf5c56a02cb8/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:35.113531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=28d0bb1d968841c2986b5eb164af953b, user_id=c4129e5f14794a619acdaf5c56a02cb8, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:35.116204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:35.183774 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1175/2361] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:25:35 2022] PUT /identity/v3/projects/28d0bb1d968841c2986b5eb164af953b/users/c4129e5f14794a619acdaf5c56a02cb8/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:35.189169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22536b6-19c9-4212-bce8-94b631a26f0e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:35.189527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22536b6-19c9-4212-bce8-94b631a26f0e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:35.192551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22536b6-19c9-4212-bce8-94b631a26f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:35.209445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e22536b6-19c9-4212-bce8-94b631a26f0e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:35.210707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e22536b6-19c9-4212-bce8-94b631a26f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:35.269176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1187/2362] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:35.278630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:35.280434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:35.337198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a90a51-a478-4602-b67a-09b0f47f8abd admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:35.340870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa87ee-969f-4353-a8dc-949711884d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:35.341472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa87ee-969f-4353-a8dc-949711884d6d service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:35.341789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa87ee-969f-4353-a8dc-949711884d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:35.342149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa87ee-969f-4353-a8dc-949711884d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:35.351235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa87ee-969f-4353-a8dc-949711884d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:35.352662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa87ee-969f-4353-a8dc-949711884d6d service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:35.363308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1176/2363] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:42.003438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc430c-42b1-41f5-ad89-0e4f555a9600 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.003438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc430c-42b1-41f5-ad89-0e4f555a9600 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.003438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc430c-42b1-41f5-ad89-0e4f555a9600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:42.010839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:42.036289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-30bc430c-42b1-41f5-ad89-0e4f555a9600 None None] MFA Rules not processed for user `c4129e5f14794a619acdaf5c56a02cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:42.037766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-30bc430c-42b1-41f5-ad89-0e4f555a9600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:42.110204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1188/2364] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:42.114846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa87ee-969f-4353-a8dc-949711884d6d service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:42.126665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa87ee-969f-4353-a8dc-949711884d6d service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:42.131666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac926694-05b1-44be-9719-2d56cc777ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:42.132554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac926694-05b1-44be-9719-2d56cc777ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.132820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac926694-05b1-44be-9719-2d56cc777ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.133086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac926694-05b1-44be-9719-2d56cc777ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:42.133780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac926694-05b1-44be-9719-2d56cc777ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:42.135915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac926694-05b1-44be-9719-2d56cc777ab8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:42.141091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:42.253885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1177/2365] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:25:42 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:25:42.257952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc430c-42b1-41f5-ad89-0e4f555a9600 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:42.274858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc430c-42b1-41f5-ad89-0e4f555a9600 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:42.353479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3a110-e399-4c4e-bb58-707158431fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:42.354127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3a110-e399-4c4e-bb58-707158431fbc admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.354436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3a110-e399-4c4e-bb58-707158431fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.354703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3a110-e399-4c4e-bb58-707158431fbc admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:42.355346 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3a110-e399-4c4e-bb58-707158431fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:42.357274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3a110-e399-4c4e-bb58-707158431fbc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:42.454491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1189/2366] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:25:42 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:25:42.456364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac926694-05b1-44be-9719-2d56cc777ab8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:42.468172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac926694-05b1-44be-9719-2d56cc777ab8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:42.472813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0d67-d169-423c-bb5c-da09ade2ff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:42.473612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0d67-d169-423c-bb5c-da09ade2ff98 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.473890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0d67-d169-423c-bb5c-da09ade2ff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.474157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f0d67-d169-423c-bb5c-da09ade2ff98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:42.474830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f0d67-d169-423c-bb5c-da09ade2ff98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:42.476427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f0d67-d169-423c-bb5c-da09ade2ff98 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:42.491386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1178/2367] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:42.492587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3a110-e399-4c4e-bb58-707158431fbc admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:42.503625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3a110-e399-4c4e-bb58-707158431fbc admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:42.508366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f2d0f-798f-4370-bdb6-55eb6b6e4746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:42.509020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f2d0f-798f-4370-bdb6-55eb6b6e4746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.509298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f2d0f-798f-4370-bdb6-55eb6b6e4746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.509565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f2d0f-798f-4370-bdb6-55eb6b6e4746 admin admin] PATH_INFO: `/v3/projects/74b5ccd665c245debc0f7bfe5a97237e/users/0e082fd6f18849029c18681675d05fb0/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:42.525782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f2d0f-798f-4370-bdb6-55eb6b6e4746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b5ccd665c245debc0f7bfe5a97237e, user_id=0e082fd6f18849029c18681675d05fb0, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:42.528535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f2d0f-798f-4370-bdb6-55eb6b6e4746 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:42.595396 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1190/2368] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:25:42 2022] PUT /identity/v3/projects/74b5ccd665c245debc0f7bfe5a97237e/users/0e082fd6f18849029c18681675d05fb0/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:42.599420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0d67-d169-423c-bb5c-da09ade2ff98 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:42.626051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f0d67-d169-423c-bb5c-da09ade2ff98 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:42.640599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a96f-6468-4c9d-a368-10c2e0397c19 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.640914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a96f-6468-4c9d-a368-10c2e0397c19 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.641191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a96f-6468-4c9d-a368-10c2e0397c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:42.669494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4f21a96f-6468-4c9d-a368-10c2e0397c19 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:42.671161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4f21a96f-6468-4c9d-a368-10c2e0397c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:42.709566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b5537-9926-4747-b33b-6e7a35e3be72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:42.710176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5537-9926-4747-b33b-6e7a35e3be72 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.710926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5537-9926-4747-b33b-6e7a35e3be72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.710926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b5537-9926-4747-b33b-6e7a35e3be72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:42.711377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b5537-9926-4747-b33b-6e7a35e3be72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:42.713187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b5537-9926-4747-b33b-6e7a35e3be72 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:42.724711 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1179/2369] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:42.729177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b5537-9926-4747-b33b-6e7a35e3be72 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:42.739730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1191/2370] 10.222.0.20 () {58 vars in 967 bytes} [Fri Sep 9 23:25:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:25:42.740973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b5537-9926-4747-b33b-6e7a35e3be72 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:42.747455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca570b02-7965-47f4-8462-3c7c92e9f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:42.749243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca570b02-7965-47f4-8462-3c7c92e9f843 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.749243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca570b02-7965-47f4-8462-3c7c92e9f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.749243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca570b02-7965-47f4-8462-3c7c92e9f843 admin admin] PATH_INFO: `/v3/projects/74b5ccd665c245debc0f7bfe5a97237e/users/0e082fd6f18849029c18681675d05fb0/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:42.751490 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca570b02-7965-47f4-8462-3c7c92e9f843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b5ccd665c245debc0f7bfe5a97237e, user_id=0e082fd6f18849029c18681675d05fb0, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:42.755050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a96f-6468-4c9d-a368-10c2e0397c19 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:42.757281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca570b02-7965-47f4-8462-3c7c92e9f843 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:42.757707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4f21a96f-6468-4c9d-a368-10c2e0397c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:42.765059 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:42.777350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1180/2371] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:25:42 2022] PUT /identity/v3/projects/74b5ccd665c245debc0f7bfe5a97237e/users/0e082fd6f18849029c18681675d05fb0/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:42.840966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a96f-6468-4c9d-a368-10c2e0397c19 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:42.951976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b4d27-5574-46cd-863d-4cc7dd268107 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:42.952596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b4d27-5574-46cd-863d-4cc7dd268107 service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:42.953050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b4d27-5574-46cd-863d-4cc7dd268107 service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:42.953216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b4d27-5574-46cd-863d-4cc7dd268107 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:43.036650 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b4d27-5574-46cd-863d-4cc7dd268107 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:43.039099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b4d27-5574-46cd-863d-4cc7dd268107 service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:43.049301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1192/2372] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:25:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 23:25:45.723666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2d32-2112-437e-b818-7857fa15a69c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:45.726383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2d32-2112-437e-b818-7857fa15a69c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:45.726383 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2d32-2112-437e-b818-7857fa15a69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:45.746501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-925d2d32-2112-437e-b818-7857fa15a69c None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:45.749286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-925d2d32-2112-437e-b818-7857fa15a69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:45.759264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1181/2373] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:25:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:45.774871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b4d27-5574-46cd-863d-4cc7dd268107 service swift] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:45.785470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b4d27-5574-46cd-863d-4cc7dd268107 service swift] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:45.790542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808350-e709-49c8-897c-073fe8b47560 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:45.791142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808350-e709-49c8-897c-073fe8b47560 service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:45.791414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808350-e709-49c8-897c-073fe8b47560 service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:45.791677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7808350-e709-49c8-897c-073fe8b47560 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:45.808348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7808350-e709-49c8-897c-073fe8b47560 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:45.808348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7808350-e709-49c8-897c-073fe8b47560 service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:45.812584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1193/2374] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:25:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:25:47.043855 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea087c-ee6f-4e65-8a59-192169f507d4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:47.044395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea087c-ee6f-4e65-8a59-192169f507d4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:47.044520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea087c-ee6f-4e65-8a59-192169f507d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:47.072377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-94ea087c-ee6f-4e65-8a59-192169f507d4 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:47.073743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-94ea087c-ee6f-4e65-8a59-192169f507d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:47.082759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1182/2375] 10.222.0.20 () {58 vars in 967 bytes} [Fri Sep 9 23:25:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:47.096640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808350-e709-49c8-897c-073fe8b47560 service swift] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:47.112270 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7808350-e709-49c8-897c-073fe8b47560 service swift] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:47.118276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557da10b-047f-485d-81db-2faf1ea09ecc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:47.119206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557da10b-047f-485d-81db-2faf1ea09ecc service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:47.119811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557da10b-047f-485d-81db-2faf1ea09ecc service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:47.119914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557da10b-047f-485d-81db-2faf1ea09ecc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:47.137440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557da10b-047f-485d-81db-2faf1ea09ecc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:47.139800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557da10b-047f-485d-81db-2faf1ea09ecc service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:47.152190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1194/2376] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:25:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:25:47.875764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4a62a-9f2a-4ffa-be9c-6175bb42265a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:47.876335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4a62a-9f2a-4ffa-be9c-6175bb42265a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:47.876765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4a62a-9f2a-4ffa-be9c-6175bb42265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:47.906192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6cb4a62a-9f2a-4ffa-be9c-6175bb42265a None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:47.907866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6cb4a62a-9f2a-4ffa-be9c-6175bb42265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:47.921074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1183/2377] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:25:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 09 23:25:47.933072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557da10b-047f-485d-81db-2faf1ea09ecc service swift] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:47.945234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557da10b-047f-485d-81db-2faf1ea09ecc service swift] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:47.951828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b0b23-2fd1-489d-bac3-d811aa546b3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:47.952496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b0b23-2fd1-489d-bac3-d811aa546b3d service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:47.952880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b0b23-2fd1-489d-bac3-d811aa546b3d service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:47.953229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b0b23-2fd1-489d-bac3-d811aa546b3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:47.962560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b0b23-2fd1-489d-bac3-d811aa546b3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:47.964293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b0b23-2fd1-489d-bac3-d811aa546b3d service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:47.974279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1195/2378] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:25:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:25:48.706883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c833416-c1ee-4f61-b122-a4da3002c1a8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:48.707407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c833416-c1ee-4f61-b122-a4da3002c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:48.707407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c833416-c1ee-4f61-b122-a4da3002c1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:48.728593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2c833416-c1ee-4f61-b122-a4da3002c1a8 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:48.729884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2c833416-c1ee-4f61-b122-a4da3002c1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:48.740992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1184/2379] 10.222.0.20 () {58 vars in 967 bytes} [Fri Sep 9 23:25:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:48.754496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b0b23-2fd1-489d-bac3-d811aa546b3d service swift] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:48.767369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b0b23-2fd1-489d-bac3-d811aa546b3d service swift] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:48.788266 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:48.788890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:48.789170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:48.789444 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:48.802436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:48.811043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:48.812723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:48.829493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1196/2380] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:25:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:25:49.297213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e190e8-dfd3-422a-b465-d85a22d00396 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:49.297497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e190e8-dfd3-422a-b465-d85a22d00396 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:49.297770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e190e8-dfd3-422a-b465-d85a22d00396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:49.332345 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-18e190e8-dfd3-422a-b465-d85a22d00396 None None] MFA Rules not processed for user `0e082fd6f18849029c18681675d05fb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:49.334049 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-18e190e8-dfd3-422a-b465-d85a22d00396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:49.424127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1185/2381] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:49.438532 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:49.440173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:49.509872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196770d-6d63-4fb1-a26c-f5824cc952df service swift] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:49.516776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a057fb-daa3-400a-b422-e8ec31678229 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1842c116caa4a5d9d75c8605eeec0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:49.516776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a057fb-daa3-400a-b422-e8ec31678229 service glance] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:49.516776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a057fb-daa3-400a-b422-e8ec31678229 service glance] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:49.516776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a057fb-daa3-400a-b422-e8ec31678229 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:49.580291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a057fb-daa3-400a-b422-e8ec31678229 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:49.581732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a057fb-daa3-400a-b422-e8ec31678229 service glance] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:49.605272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1197/2382] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:49.842542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e37b-1aaa-4354-ac70-63051c31237c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:49.842542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e37b-1aaa-4354-ac70-63051c31237c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:49.842542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071e37b-1aaa-4354-ac70-63051c31237c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:49.863682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a071e37b-1aaa-4354-ac70-63051c31237c None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:49.864912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a071e37b-1aaa-4354-ac70-63051c31237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:49.872224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1186/2383] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:25:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:49.885401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a057fb-daa3-400a-b422-e8ec31678229 service glance] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:49.897482 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a057fb-daa3-400a-b422-e8ec31678229 service glance] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:49.902463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893e2f-b0e8-4b29-a807-0683ec25cdac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:49.903060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893e2f-b0e8-4b29-a807-0683ec25cdac service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:49.903348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893e2f-b0e8-4b29-a807-0683ec25cdac service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:49.903612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893e2f-b0e8-4b29-a807-0683ec25cdac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:49.912368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09893e2f-b0e8-4b29-a807-0683ec25cdac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:49.917098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09893e2f-b0e8-4b29-a807-0683ec25cdac service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:49.922125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1198/2384] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:25:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:25:51.981251 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071e37b-1aaa-4354-ac70-63051c31237c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:51.983878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a071e37b-1aaa-4354-ac70-63051c31237c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:52.004181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:52.095589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071e37b-1aaa-4354-ac70-63051c31237c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:52.101395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:52.104808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:52.104808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:52.104808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] PATH_INFO: `/v3/users/4398271cfc094e81b7a120bdeeb6dafe` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:52.104808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=4398271cfc094e81b7a120bdeeb6dafe)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:52.106885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:52.295252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4398271cfc094e81b7a120bdeeb6dafe'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:52.388959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4398271cfc094e81b7a120bdeeb6dafe'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:52.389835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4398271cfc094e81b7a120bdeeb6dafe'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:52.398715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4398271cfc094e81b7a120bdeeb6dafe'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:52.465428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4398271cfc094e81b7a120bdeeb6dafe'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:52.474818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1187/2385] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:25:51 2022] DELETE /identity/v3/users/4398271cfc094e81b7a120bdeeb6dafe => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:52.479132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9792e1c-ab69-4ace-84a3-44a08e581ada None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:52.480052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9792e1c-ab69-4ace-84a3-44a08e581ada None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:52.480677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9792e1c-ab69-4ace-84a3-44a08e581ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:52.505432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a9792e1c-ab69-4ace-84a3-44a08e581ada None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:52.507441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a9792e1c-ab69-4ace-84a3-44a08e581ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:52.590495 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1199/2386] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:52.604475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:52.624394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:52.729190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862c25d-95ae-4973-8865-20a0c6232ece admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:52.734096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e605c767-961b-4be9-a9ff-b38adba0a4ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:52.738823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605c767-961b-4be9-a9ff-b38adba0a4ea service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:52.738823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605c767-961b-4be9-a9ff-b38adba0a4ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:52.738823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605c767-961b-4be9-a9ff-b38adba0a4ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:52.768898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605c767-961b-4be9-a9ff-b38adba0a4ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:52.770276 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e605c767-961b-4be9-a9ff-b38adba0a4ea service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:52.785570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1188/2387] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:25:53.219671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9792e1c-ab69-4ace-84a3-44a08e581ada None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:53.221826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a9792e1c-ab69-4ace-84a3-44a08e581ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:53.240594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9792e1c-ab69-4ace-84a3-44a08e581ada None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:53.247098 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:53.248297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:53.248891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:53.249150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] PATH_INFO: `/v3/projects/d4694a14825f4600a0336e150452a9cf` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:53.250670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4694a14825f4600a0336e150452a9cf)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:53.252284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:53.385839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:53.400715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4694a14825f4600a0336e150452a9cf', 'request_id': 'req-bbdc1746-d790-4dea-b7c5-77a93214da75'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:53.468663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4694a14825f4600a0336e150452a9cf', 'request_id': 'req-bbdc1746-d790-4dea-b7c5-77a93214da75'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:53.480193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4694a14825f4600a0336e150452a9cf', 'request_id': 'req-bbdc1746-d790-4dea-b7c5-77a93214da75'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:53.500173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] The token cache is being invalidate because project d4694a14825f4600a0336e150452a9cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:25:53.500470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:53.501376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bbdc1746-d790-4dea-b7c5-77a93214da75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:25:53.503558 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1200/2388] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:25:53 2022] DELETE /identity/v3/projects/d4694a14825f4600a0336e150452a9cf => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:53.508476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cbe48-807d-4753-8239-132ee560e5af None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:53.509313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cbe48-807d-4753-8239-132ee560e5af None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:53.509587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cbe48-807d-4753-8239-132ee560e5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:53.534480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4f2cbe48-807d-4753-8239-132ee560e5af None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:53.535997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4f2cbe48-807d-4753-8239-132ee560e5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:53.570470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:25:53.614047 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1189/2389] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:53.622627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c4fc-c7dd-4c8b-b689-ff8a6c9bb36c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:53.622941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c4fc-c7dd-4c8b-b689-ff8a6c9bb36c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:53.623567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6c4fc-c7dd-4c8b-b689-ff8a6c9bb36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:53.648764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-34e6c4fc-c7dd-4c8b-b689-ff8a6c9bb36c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:53.650703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-34e6c4fc-c7dd-4c8b-b689-ff8a6c9bb36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:53.661379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1201/2390] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:53.665479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cbe48-807d-4753-8239-132ee560e5af None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:53.681459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cbe48-807d-4753-8239-132ee560e5af None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:53.693994 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:53.694823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:53.695206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:53.695572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:53.696333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:25:53.696741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:53.700176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:53.716926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1190/2391] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:25:53.721718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6c4fc-c7dd-4c8b-b689-ff8a6c9bb36c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:53.731294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6c4fc-c7dd-4c8b-b689-ff8a6c9bb36c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:53.745300 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf977be6-ce61-4c68-8a25-8a67d6d516ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:53.746370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf977be6-ce61-4c68-8a25-8a67d6d516ba admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:53.746701 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf977be6-ce61-4c68-8a25-8a67d6d516ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:53.746977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf977be6-ce61-4c68-8a25-8a67d6d516ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:53.748184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf977be6-ce61-4c68-8a25-8a67d6d516ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:53.751914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf977be6-ce61-4c68-8a25-8a67d6d516ba admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:53.866403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1202/2392] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:25:53 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:25:53.869790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:53.882630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c81ec-4558-47bc-8781-20611d3c49a5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:53.979091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bd6a6-604f-4a6f-b7c0-7212d08ea67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:53.981580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bd6a6-604f-4a6f-b7c0-7212d08ea67b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:53.981580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bd6a6-604f-4a6f-b7c0-7212d08ea67b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:53.981580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bd6a6-604f-4a6f-b7c0-7212d08ea67b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:53.981580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bd6a6-604f-4a6f-b7c0-7212d08ea67b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:53.983866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bd6a6-604f-4a6f-b7c0-7212d08ea67b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:54.065618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf977be6-ce61-4c68-8a25-8a67d6d516ba admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:54.066734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1191/2393] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:25:53 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:25:54.081131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf977be6-ce61-4c68-8a25-8a67d6d516ba admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:54.086732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd9e57-861b-4bfa-bb42-1a186feb66c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:54.087523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9e57-861b-4bfa-bb42-1a186feb66c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:54.087804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9e57-861b-4bfa-bb42-1a186feb66c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:54.088070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd9e57-861b-4bfa-bb42-1a186feb66c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:54.088700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd9e57-861b-4bfa-bb42-1a186feb66c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:54.090435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd9e57-861b-4bfa-bb42-1a186feb66c5 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:54.103638 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1203/2394] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:54.108700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bd6a6-604f-4a6f-b7c0-7212d08ea67b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:54.122449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bd6a6-604f-4a6f-b7c0-7212d08ea67b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:54.129184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af08ae-bfd8-4e56-8283-7af36fbae787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:54.130057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af08ae-bfd8-4e56-8283-7af36fbae787 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:54.130384 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af08ae-bfd8-4e56-8283-7af36fbae787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:54.130662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af08ae-bfd8-4e56-8283-7af36fbae787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:54.131326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af08ae-bfd8-4e56-8283-7af36fbae787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:54.132919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af08ae-bfd8-4e56-8283-7af36fbae787 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:54.146947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1192/2395] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:25:54.151821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd9e57-861b-4bfa-bb42-1a186feb66c5 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:54.163613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd9e57-861b-4bfa-bb42-1a186feb66c5 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:54.167748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:54.168401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:54.168671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:54.168936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] PATH_INFO: `/v3/projects/602875e689b7476697ce2e9bbc3bf7b5/users/cf9d4a796a8c450486832bb10c3d4ad0/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:54.187710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] RBAC: Authorizing `identity:create_grant(project_id=602875e689b7476697ce2e9bbc3bf7b5, user_id=cf9d4a796a8c450486832bb10c3d4ad0, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:54.190412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:54.264721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1204/2396] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:25:54 2022] PUT /identity/v3/projects/602875e689b7476697ce2e9bbc3bf7b5/users/cf9d4a796a8c450486832bb10c3d4ad0/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:25:54.270575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb75292-4a9f-4b4f-8672-96ca2fff2cff None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:54.270575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb75292-4a9f-4b4f-8672-96ca2fff2cff None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:54.270575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb75292-4a9f-4b4f-8672-96ca2fff2cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:54.292899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4cb75292-4a9f-4b4f-8672-96ca2fff2cff None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:25:54.294207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4cb75292-4a9f-4b4f-8672-96ca2fff2cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:54.363069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1193/2397] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:25:54.372336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:25:54.374355 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:25:54.448375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e838eb-d544-4d4e-be49-d6288f7aa41b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:25:54.455333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0bb358-f2cb-42f1-8657-1941e1b980e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:25:54.456104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bb358-f2cb-42f1-8657-1941e1b980e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:25:54.456490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bb358-f2cb-42f1-8657-1941e1b980e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:25:54.456868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bb358-f2cb-42f1-8657-1941e1b980e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:25:54.467035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0bb358-f2cb-42f1-8657-1941e1b980e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:25:54.468557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0bb358-f2cb-42f1-8657-1941e1b980e7 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:25:54.482902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1205/2398] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:01.277486 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513bb64-a541-47b9-86ff-f11ae39f3c51 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:01.277978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513bb64-a541-47b9-86ff-f11ae39f3c51 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:01.278149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513bb64-a541-47b9-86ff-f11ae39f3c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:01.328167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8513bb64-a541-47b9-86ff-f11ae39f3c51 None None] MFA Rules not processed for user `cf9d4a796a8c450486832bb10c3d4ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:01.329533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8513bb64-a541-47b9-86ff-f11ae39f3c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:01.365773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:01.413684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1194/2399] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:01.418250 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0bb358-f2cb-42f1-8657-1941e1b980e7 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:01.428609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0bb358-f2cb-42f1-8657-1941e1b980e7 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:01.433215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018e23f-8430-4416-8498-239948e74e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:01.433864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018e23f-8430-4416-8498-239948e74e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:01.434154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018e23f-8430-4416-8498-239948e74e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:01.434443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018e23f-8430-4416-8498-239948e74e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:01.435645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018e23f-8430-4416-8498-239948e74e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:01.437752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018e23f-8430-4416-8498-239948e74e22 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:01.521936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1206/2400] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:26:01 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:26:01.525905 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513bb64-a541-47b9-86ff-f11ae39f3c51 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:01.537734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513bb64-a541-47b9-86ff-f11ae39f3c51 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:01.600818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c29f8-ac67-4a55-a8a7-c572255304a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:01.601432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c29f8-ac67-4a55-a8a7-c572255304a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:01.601709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c29f8-ac67-4a55-a8a7-c572255304a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:01.601984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c29f8-ac67-4a55-a8a7-c572255304a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:01.602674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c29f8-ac67-4a55-a8a7-c572255304a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:01.604955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c29f8-ac67-4a55-a8a7-c572255304a2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:01.692041 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1195/2401] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:26:01 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:26:01.696111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018e23f-8430-4416-8498-239948e74e22 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:01.707889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018e23f-8430-4416-8498-239948e74e22 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:01.712705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3580f-9b57-429b-a397-e69f34ffeede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:01.713313 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3580f-9b57-429b-a397-e69f34ffeede admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:01.713589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3580f-9b57-429b-a397-e69f34ffeede admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:01.713859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3580f-9b57-429b-a397-e69f34ffeede admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:01.714529 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3580f-9b57-429b-a397-e69f34ffeede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:01.716103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3580f-9b57-429b-a397-e69f34ffeede admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:01.728235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1207/2402] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:26:01.732956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c29f8-ac67-4a55-a8a7-c572255304a2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:01.743961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c29f8-ac67-4a55-a8a7-c572255304a2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:01.753785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcbd75-20d6-4d53-a406-e48775ce8567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:01.754478 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbd75-20d6-4d53-a406-e48775ce8567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:01.754760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbd75-20d6-4d53-a406-e48775ce8567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:01.755036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcbd75-20d6-4d53-a406-e48775ce8567 admin admin] PATH_INFO: `/v3/projects/87c007e6d40d40e3b268dfd2fe3a0d7d/users/9c26d229acd5487e807bd74eaaaba535/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:01.777171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcbd75-20d6-4d53-a406-e48775ce8567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c007e6d40d40e3b268dfd2fe3a0d7d, user_id=9c26d229acd5487e807bd74eaaaba535, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:01.779885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfcbd75-20d6-4d53-a406-e48775ce8567 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:01.854184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1196/2403] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:26:01 2022] PUT /identity/v3/projects/87c007e6d40d40e3b268dfd2fe3a0d7d/users/9c26d229acd5487e807bd74eaaaba535/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:01.858186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3580f-9b57-429b-a397-e69f34ffeede admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:01.869472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3580f-9b57-429b-a397-e69f34ffeede admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:01.895023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:01.930699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a183118-437a-4873-9df2-029d54548518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:01.931305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183118-437a-4873-9df2-029d54548518 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:01.931601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183118-437a-4873-9df2-029d54548518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:01.931850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a183118-437a-4873-9df2-029d54548518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:01.932517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a183118-437a-4873-9df2-029d54548518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:01.934136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a183118-437a-4873-9df2-029d54548518 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:01.944089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1208/2404] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:26:01.948730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcbd75-20d6-4d53-a406-e48775ce8567 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:01.960252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcbd75-20d6-4d53-a406-e48775ce8567 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:01.965629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af0c58-63b1-46f2-83c2-37ccde83f983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:01.967453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af0c58-63b1-46f2-83c2-37ccde83f983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:01.967736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af0c58-63b1-46f2-83c2-37ccde83f983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:01.968006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af0c58-63b1-46f2-83c2-37ccde83f983 admin admin] PATH_INFO: `/v3/projects/87c007e6d40d40e3b268dfd2fe3a0d7d/users/9c26d229acd5487e807bd74eaaaba535/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:01.970096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af0c58-63b1-46f2-83c2-37ccde83f983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c007e6d40d40e3b268dfd2fe3a0d7d, user_id=9c26d229acd5487e807bd74eaaaba535, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:01.972639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af0c58-63b1-46f2-83c2-37ccde83f983 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:01.984079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1197/2405] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:26:01 2022] PUT /identity/v3/projects/87c007e6d40d40e3b268dfd2fe3a0d7d/users/9c26d229acd5487e807bd74eaaaba535/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:02.914710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26897591-fde1-4a97-81ce-a5bbabefdb4d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:02.915272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26897591-fde1-4a97-81ce-a5bbabefdb4d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:02.915272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26897591-fde1-4a97-81ce-a5bbabefdb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:02.940936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-26897591-fde1-4a97-81ce-a5bbabefdb4d None None] MFA Rules not processed for user `c4129e5f14794a619acdaf5c56a02cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:02.942254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-26897591-fde1-4a97-81ce-a5bbabefdb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:03.012851 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1209/2406] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:07.084439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ada8-0976-491d-950b-a792126fd2bc None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:07.085379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ada8-0976-491d-950b-a792126fd2bc None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:07.086538 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448ada8-0976-491d-950b-a792126fd2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:07.117597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9448ada8-0976-491d-950b-a792126fd2bc None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:07.121929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9448ada8-0976-491d-950b-a792126fd2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:07.247123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1198/2407] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:26:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:07.262955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26897591-fde1-4a97-81ce-a5bbabefdb4d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:07.265481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-26897591-fde1-4a97-81ce-a5bbabefdb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:07.398618 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26897591-fde1-4a97-81ce-a5bbabefdb4d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:07.404623 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab6f79-9110-4d57-a3c5-6648d8be59b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:07.405564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab6f79-9110-4d57-a3c5-6648d8be59b0 service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:07.405936 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab6f79-9110-4d57-a3c5-6648d8be59b0 service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:07.406302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab6f79-9110-4d57-a3c5-6648d8be59b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:07.415522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab6f79-9110-4d57-a3c5-6648d8be59b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:07.417042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab6f79-9110-4d57-a3c5-6648d8be59b0 service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:07.426713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1210/2408] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:26:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:26:08.410872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448ada8-0976-491d-950b-a792126fd2bc None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:08.526752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:08.593432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448ada8-0976-491d-950b-a792126fd2bc None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:08.603484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:08.605795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:08.607697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:08.609087 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:08.618758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:08.767857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdde84-6dae-4dbd-8470-82481327fd71 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:08.768352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdde84-6dae-4dbd-8470-82481327fd71 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:08.768730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fdde84-6dae-4dbd-8470-82481327fd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:08.823196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:08.824837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:08.835404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-08fdde84-6dae-4dbd-8470-82481327fd71 None None] MFA Rules not processed for user `9c26d229acd5487e807bd74eaaaba535`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:08.837913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-08fdde84-6dae-4dbd-8470-82481327fd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:08.845382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1199/2409] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:08.952114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:08.997815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1211/2410] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:09.011135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:09.012679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:09.031447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e139fa-6e98-4160-a03c-d899e378743f service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:09.039141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:09.039823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:09.040447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:09.040784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:09.155662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:09.157235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:09.171209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1200/2411] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:26:12.324370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c22405-e33f-4941-8dc3-a70042262104 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:12.324864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c22405-e33f-4941-8dc3-a70042262104 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:12.324953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c22405-e33f-4941-8dc3-a70042262104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:12.350871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-d4c22405-e33f-4941-8dc3-a70042262104 None None] MFA Rules not processed for user `cf9d4a796a8c450486832bb10c3d4ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:12.352772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d4c22405-e33f-4941-8dc3-a70042262104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:12.365106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1212/2412] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:12.825986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:12.827838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:12.850743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea08022-8c8e-4431-bf3a-ce438a549733 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:12.862061 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:12.862802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:12.863170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:12.863516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:12.865857 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:12.891622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:12.893127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:12.905986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1201/2413] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:20.509793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c22405-e33f-4941-8dc3-a70042262104 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:20.523336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c22405-e33f-4941-8dc3-a70042262104 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:20.528921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:20.529510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:20.529787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:20.530055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:20.531846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:20.599960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:20.601639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:20.612585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1213/2414] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:21.240986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:21.250046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2622ab-6ccc-4dce-8edd-03a468dc0785 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:21.254079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:21.254829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:21.255117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:21.255391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:21.257103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:21.316809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:21.345662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:21.348213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:21.360082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1202/2415] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:29.652001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:29.662285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188e526-a090-4a75-83e2-3e23950f0ff1 service nova] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:29.665955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8551a1-6dcc-4da5-b48f-b981e224c685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:29.666574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8551a1-6dcc-4da5-b48f-b981e224c685 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:29.666848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8551a1-6dcc-4da5-b48f-b981e224c685 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:29.667111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8551a1-6dcc-4da5-b48f-b981e224c685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:29.668828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ec8551a1-6dcc-4da5-b48f-b981e224c685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:29.678912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8551a1-6dcc-4da5-b48f-b981e224c685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:29.680521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8551a1-6dcc-4da5-b48f-b981e224c685 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:29.689419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1214/2416] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:30.515078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:30.518940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:30.605016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cef3b-1523-44d7-9961-eb4ab9cc4607 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:30.611573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:30.612439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:30.613014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:30.613499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:30.616761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:30.631603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:30.633477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:30.681645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1203/2417] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:34.489633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834dd4f-46c4-4df2-86bd-1409791107d7 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:34.490156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834dd4f-46c4-4df2-86bd-1409791107d7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:34.490256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834dd4f-46c4-4df2-86bd-1409791107d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:34.513862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3834dd4f-46c4-4df2-86bd-1409791107d7 None None] MFA Rules not processed for user `f097c4fc270748cc8d41d7bcaba672a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:34.515581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3834dd4f-46c4-4df2-86bd-1409791107d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:34.526739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1215/2418] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:34.967051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:34.977024 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb2df6-7716-48e7-932c-af285f9a16f8 service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:34.981078 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:34.981674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:34.981944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:34.982235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:34.984667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:35.013208 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:35.015716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:35.037339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1204/2419] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:35.289645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834dd4f-46c4-4df2-86bd-1409791107d7 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:35.303727 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834dd4f-46c4-4df2-86bd-1409791107d7 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:35.330314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147bf874-f008-46ff-a1a5-f8a971167017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:35.331161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bf874-f008-46ff-a1a5-f8a971167017 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:35.331407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bf874-f008-46ff-a1a5-f8a971167017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:35.332192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147bf874-f008-46ff-a1a5-f8a971167017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:35.334598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-147bf874-f008-46ff-a1a5-f8a971167017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:35.349150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147bf874-f008-46ff-a1a5-f8a971167017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:35.350889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147bf874-f008-46ff-a1a5-f8a971167017 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:35.375044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1216/2420] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:36.384200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:36.400708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a49ef-b674-4967-886c-9ecdeb3fd258 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:36.407162 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:36.407819 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:36.408098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:36.408365 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:36.410284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:36.427152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:36.428778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:36.440216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1205/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:38.704548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c6c5-5a7a-4fc5-b998-f6fd3a33635a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:38.705075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c6c5-5a7a-4fc5-b998-f6fd3a33635a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:38.705204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80c6c5-5a7a-4fc5-b998-f6fd3a33635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:38.730112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7f80c6c5-5a7a-4fc5-b998-f6fd3a33635a None None] MFA Rules not processed for user `27be0979440d4ab081dae31790280eff`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:38.731609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7f80c6c5-5a7a-4fc5-b998-f6fd3a33635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:38.824359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1217/2422] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:39.229663 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:39.231839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:39.248473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3545b4-ccc5-4338-a5e1-a1c444e4618e service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:39.255928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:39.256898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:39.257427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:39.257829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:39.259799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:39.275073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:39.276938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:39.298761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1206/2423] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:45.554428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80c6c5-5a7a-4fc5-b998-f6fd3a33635a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:45.559784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=80 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:45.566426 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80c6c5-5a7a-4fc5-b998-f6fd3a33635a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:45.572675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:45.573632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:45.576330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:45.577393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:45.579885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:45.649029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:45.650518 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:45.660817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1218/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:48.481714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:48.492106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a581e6-edaf-46a0-94b9-1572d6446b08 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:48.495731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:48.496337 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:48.496614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:48.496878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:48.498917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:48.576779 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:48.580683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:48.592639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1207/2425] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:48.926005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:48.943054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d43fdc-c658-4d86-8c58-c8d876dbcba7 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:48.951062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:48.954805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:48.954805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:48.954805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:48.955212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:48.971441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:48.977645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:49.013364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1219/2426] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:52.039601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:52.043815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=86 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:52.049511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cc448-bc78-41f5-b3af-5f6bf60145ff service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:52.053337 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388657a4-6ef4-41b1-832f-d1dd7b69d44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:52.053964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388657a4-6ef4-41b1-832f-d1dd7b69d44e service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:52.054243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388657a4-6ef4-41b1-832f-d1dd7b69d44e service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:52.054657 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388657a4-6ef4-41b1-832f-d1dd7b69d44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:52.065659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388657a4-6ef4-41b1-832f-d1dd7b69d44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:52.067364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388657a4-6ef4-41b1-832f-d1dd7b69d44e service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:52.079891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1208/2427] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:54.478438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:54.480307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:54.505843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4a84b-9e22-44cc-9e0c-92386c63b75a service nova] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:54.521970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:54.521970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:54.521970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:54.521970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] PATH_INFO: `/v3/users/987d2f6bb1ff4f1fa248c3ff410c8119` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:54.522393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=987d2f6bb1ff4f1fa248c3ff410c8119)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:54.524337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:54.818389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987d2f6bb1ff4f1fa248c3ff410c8119'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:54.819615 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987d2f6bb1ff4f1fa248c3ff410c8119'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:54.901993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987d2f6bb1ff4f1fa248c3ff410c8119'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:54.919710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987d2f6bb1ff4f1fa248c3ff410c8119'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:54.928233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-c8a944a2-06ad-45d5-a045-29419c6f59bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987d2f6bb1ff4f1fa248c3ff410c8119'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:54.939809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1220/2428] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:26:54 2022] DELETE /identity/v3/users/987d2f6bb1ff4f1fa248c3ff410c8119 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:55.368449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388657a4-6ef4-41b1-832f-d1dd7b69d44e service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:55.388186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388657a4-6ef4-41b1-832f-d1dd7b69d44e service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:55.518095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:55.518832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:55.519104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:55.519368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] PATH_INFO: `/v3/projects/57759aec45f74a689727fca5fe201e40` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:55.520519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57759aec45f74a689727fca5fe201e40)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:55.522510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:55.640376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57759aec45f74a689727fca5fe201e40', 'request_id': 'req-3d537703-816f-459c-8cde-fadf4e101017'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:55.719881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57759aec45f74a689727fca5fe201e40', 'request_id': 'req-3d537703-816f-459c-8cde-fadf4e101017'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:55.727006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57759aec45f74a689727fca5fe201e40', 'request_id': 'req-3d537703-816f-459c-8cde-fadf4e101017'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:55.744573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] The token cache is being invalidate because project 57759aec45f74a689727fca5fe201e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:26:55.745190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:55.746069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3d537703-816f-459c-8cde-fadf4e101017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:55.748007 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1209/2429] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:26:55 2022] DELETE /identity/v3/projects/57759aec45f74a689727fca5fe201e40 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:55.755136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ca69-814b-40c9-b813-6d77f55311bc None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:55.755705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ca69-814b-40c9-b813-6d77f55311bc None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:55.755705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ca69-814b-40c9-b813-6d77f55311bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:55.776616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5765ca69-814b-40c9-b813-6d77f55311bc None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:55.778268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5765ca69-814b-40c9-b813-6d77f55311bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:55.845851 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:55.862104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1221/2430] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:55.870415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740141e8-dc7f-4fb7-aacb-b832807cb787 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:55.870767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740141e8-dc7f-4fb7-aacb-b832807cb787 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:55.871154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740141e8-dc7f-4fb7-aacb-b832807cb787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:55.892250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-740141e8-dc7f-4fb7-aacb-b832807cb787 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:55.893722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-740141e8-dc7f-4fb7-aacb-b832807cb787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:55.902871 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1210/2431] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:26:55.907679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ca69-814b-40c9-b813-6d77f55311bc None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:55.922762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ca69-814b-40c9-b813-6d77f55311bc None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:55.924681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:55.925378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:55.925723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:55.926058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:55.926956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:26:55.927540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:55.929238 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:55.950415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1222/2432] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:26:55.952247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740141e8-dc7f-4fb7-aacb-b832807cb787 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:55.965327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740141e8-dc7f-4fb7-aacb-b832807cb787 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:55.977177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a877301-1714-4bad-84e3-aa06786c5d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:55.978095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a877301-1714-4bad-84e3-aa06786c5d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:55.978875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a877301-1714-4bad-84e3-aa06786c5d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:55.979297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a877301-1714-4bad-84e3-aa06786c5d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:55.980279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a877301-1714-4bad-84e3-aa06786c5d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:55.984400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a877301-1714-4bad-84e3-aa06786c5d73 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:56.088678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1211/2433] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:26:55 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:26:56.094708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:56.106545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e067e-26e0-4cd5-affc-a06ce99380d8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:56.180116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318de72-6c5b-4bbf-9738-cf280089d6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:56.181161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318de72-6c5b-4bbf-9738-cf280089d6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:56.181671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318de72-6c5b-4bbf-9738-cf280089d6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:56.182297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318de72-6c5b-4bbf-9738-cf280089d6a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:56.183967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318de72-6c5b-4bbf-9738-cf280089d6a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:56.186283 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318de72-6c5b-4bbf-9738-cf280089d6a7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:56.293475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1223/2434] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:26:56 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:26:56.293817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a877301-1714-4bad-84e3-aa06786c5d73 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:56.315035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a877301-1714-4bad-84e3-aa06786c5d73 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:56.319937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a064-d9b6-4053-beb6-e573e2843d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:56.320672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a064-d9b6-4053-beb6-e573e2843d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:56.321043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a064-d9b6-4053-beb6-e573e2843d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:56.321407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a064-d9b6-4053-beb6-e573e2843d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:56.322141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2a064-d9b6-4053-beb6-e573e2843d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:56.324576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2a064-d9b6-4053-beb6-e573e2843d81 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:56.338529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1212/2435] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:26:56.344402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318de72-6c5b-4bbf-9738-cf280089d6a7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:56.359951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318de72-6c5b-4bbf-9738-cf280089d6a7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:56.365258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d318c-3357-413f-ab2c-bd60bcb14ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:56.365967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d318c-3357-413f-ab2c-bd60bcb14ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:56.366235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d318c-3357-413f-ab2c-bd60bcb14ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:56.367170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d318c-3357-413f-ab2c-bd60bcb14ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:56.367957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d318c-3357-413f-ab2c-bd60bcb14ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:56.369620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d318c-3357-413f-ab2c-bd60bcb14ca0 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:56.393392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1224/2436] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:26:56.398574 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a064-d9b6-4053-beb6-e573e2843d81 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:56.412760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a064-d9b6-4053-beb6-e573e2843d81 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:56.418333 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:56.418927 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:56.419198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:56.419467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] PATH_INFO: `/v3/projects/09fd57fff2274a8db0c953747f6091a8/users/ced067c9eab7411988b0845aee52dc93/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:56.435868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09fd57fff2274a8db0c953747f6091a8, user_id=ced067c9eab7411988b0845aee52dc93, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:56.439902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:56.507256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1213/2437] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:26:56 2022] PUT /identity/v3/projects/09fd57fff2274a8db0c953747f6091a8/users/ced067c9eab7411988b0845aee52dc93/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:56.520908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ba94-30bc-468b-82a7-4573a6590238 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:56.520908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ba94-30bc-468b-82a7-4573a6590238 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:56.520908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55ba94-30bc-468b-82a7-4573a6590238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:56.556954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0f55ba94-30bc-468b-82a7-4573a6590238 None None] MFA Rules not processed for user `ced067c9eab7411988b0845aee52dc93`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:56.564857 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0f55ba94-30bc-468b-82a7-4573a6590238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:56.698252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1225/2438] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:56.711161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:56.712903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:56.738630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:56.815489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397991-d82b-474c-b9b5-bd9da6afb7b8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:56.823455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f0ca0-3cfa-4199-80db-024065fa1c09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:56.829513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0ca0-3cfa-4199-80db-024065fa1c09 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:56.829513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0ca0-3cfa-4199-80db-024065fa1c09 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:56.829513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f0ca0-3cfa-4199-80db-024065fa1c09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:56.834982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f0ca0-3cfa-4199-80db-024065fa1c09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:56.836891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f0ca0-3cfa-4199-80db-024065fa1c09 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:56.856878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1214/2439] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:57.151039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55ba94-30bc-468b-82a7-4573a6590238 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:57.211239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:57.230303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55ba94-30bc-468b-82a7-4573a6590238 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:57.236575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:57.237728 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:57.238260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:57.238776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] PATH_INFO: `/v3/users/ced067c9eab7411988b0845aee52dc93` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:57.240376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced067c9eab7411988b0845aee52dc93)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:57.242520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:57.414579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced067c9eab7411988b0845aee52dc93'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:57.415382 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced067c9eab7411988b0845aee52dc93'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:57.489972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced067c9eab7411988b0845aee52dc93'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:57.502878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced067c9eab7411988b0845aee52dc93'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:57.509768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced067c9eab7411988b0845aee52dc93'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:57.516333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1226/2440] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:26:57 2022] DELETE /identity/v3/users/ced067c9eab7411988b0845aee52dc93 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:57.521587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfdd89-85c5-4e8b-bb58-131b762f0cbe None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:57.522080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfdd89-85c5-4e8b-bb58-131b762f0cbe None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:57.522192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfdd89-85c5-4e8b-bb58-131b762f0cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:57.541917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-cbbfdd89-85c5-4e8b-bb58-131b762f0cbe None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:57.543157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cbbfdd89-85c5-4e8b-bb58-131b762f0cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:57.603952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1215/2441] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:57.613993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:57.615630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:57.682446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6cf30-d9f9-40a3-bf99-17d6c70b6543 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:57.686070 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9dc49b-3570-4a1c-bb98-f0ee1687c956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:57.686992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc49b-3570-4a1c-bb98-f0ee1687c956 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:57.687268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc49b-3570-4a1c-bb98-f0ee1687c956 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:57.687534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9dc49b-3570-4a1c-bb98-f0ee1687c956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:57.696628 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9dc49b-3570-4a1c-bb98-f0ee1687c956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:57.698118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9dc49b-3570-4a1c-bb98-f0ee1687c956 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:57.709704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1227/2442] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:26:58.108786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfdd89-85c5-4e8b-bb58-131b762f0cbe None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:58.110650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cbbfdd89-85c5-4e8b-bb58-131b762f0cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:58.125852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfdd89-85c5-4e8b-bb58-131b762f0cbe None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:58.130928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:58.131614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.131893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.132168 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] PATH_INFO: `/v3/projects/09fd57fff2274a8db0c953747f6091a8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.133317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09fd57fff2274a8db0c953747f6091a8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:58.135364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:58.235752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09fd57fff2274a8db0c953747f6091a8', 'request_id': 'req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:58.324696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09fd57fff2274a8db0c953747f6091a8', 'request_id': 'req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:58.332531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09fd57fff2274a8db0c953747f6091a8', 'request_id': 'req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:58.345161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] The token cache is being invalidate because project 09fd57fff2274a8db0c953747f6091a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:26:58.345551 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:58.346275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e4c425e8-37f1-4f00-a2a3-0e9c4c3db352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:26:58.347971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1216/2443] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:26:58 2022] DELETE /identity/v3/projects/09fd57fff2274a8db0c953747f6091a8 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:58.353150 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e85a3-9ac2-4c69-be0a-c27e58c9ef4b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.353630 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e85a3-9ac2-4c69-be0a-c27e58c9ef4b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.353744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e85a3-9ac2-4c69-be0a-c27e58c9ef4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.375274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-cc3e85a3-9ac2-4c69-be0a-c27e58c9ef4b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:58.376411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cc3e85a3-9ac2-4c69-be0a-c27e58c9ef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:58.446770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1228/2444] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:58.455754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ef479-9326-4c97-9c0b-dffa9df75fcd None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.456040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ef479-9326-4c97-9c0b-dffa9df75fcd None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.459718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ef479-9326-4c97-9c0b-dffa9df75fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.479308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e31ef479-9326-4c97-9c0b-dffa9df75fcd None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:58.480986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e31ef479-9326-4c97-9c0b-dffa9df75fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:58.492658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1217/2445] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:58.494692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e85a3-9ac2-4c69-be0a-c27e58c9ef4b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:58.504842 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e85a3-9ac2-4c69-be0a-c27e58c9ef4b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:58.509956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:58.510604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.510886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.511157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.511886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:26:58.512186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:58.513789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:58.524178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:58.531589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1229/2446] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:26:58.532724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ef479-9326-4c97-9c0b-dffa9df75fcd None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:58.542458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ef479-9326-4c97-9c0b-dffa9df75fcd None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:58.548380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6488a-d35e-4258-8cd4-f881666869f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:58.549073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6488a-d35e-4258-8cd4-f881666869f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.549531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6488a-d35e-4258-8cd4-f881666869f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.549876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6488a-d35e-4258-8cd4-f881666869f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.550639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6488a-d35e-4258-8cd4-f881666869f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:58.552543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6488a-d35e-4258-8cd4-f881666869f1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:58.571063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:58.646704 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1218/2447] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:26:58 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:26:58.650992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:58.661289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a62b8d-d400-4236-94d8-8627dfa9d4ba admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:58.721584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab29088-1f60-45d8-a2d6-25740931d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:58.722220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab29088-1f60-45d8-a2d6-25740931d5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.722535 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab29088-1f60-45d8-a2d6-25740931d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.724367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab29088-1f60-45d8-a2d6-25740931d5cf admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.724367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab29088-1f60-45d8-a2d6-25740931d5cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:58.725664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab29088-1f60-45d8-a2d6-25740931d5cf admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:58.816252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1230/2448] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:26:58 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:26:58.820766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6488a-d35e-4258-8cd4-f881666869f1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:58.831572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6488a-d35e-4258-8cd4-f881666869f1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:58.836225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ce847-c499-4d61-87be-e8dc3119e2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:58.836863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce847-c499-4d61-87be-e8dc3119e2fa admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.837139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce847-c499-4d61-87be-e8dc3119e2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.837412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce847-c499-4d61-87be-e8dc3119e2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.838043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ce847-c499-4d61-87be-e8dc3119e2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:58.839671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ce847-c499-4d61-87be-e8dc3119e2fa admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:58.850632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1219/2449] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:26:58.854516 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab29088-1f60-45d8-a2d6-25740931d5cf admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:58.865154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab29088-1f60-45d8-a2d6-25740931d5cf admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:58.869632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b646dc3-bf0f-4131-bcc5-9f8efc7b2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:58.870239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646dc3-bf0f-4131-bcc5-9f8efc7b2b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.870517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646dc3-bf0f-4131-bcc5-9f8efc7b2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.870810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b646dc3-bf0f-4131-bcc5-9f8efc7b2b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.871429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b646dc3-bf0f-4131-bcc5-9f8efc7b2b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:58.872990 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b646dc3-bf0f-4131-bcc5-9f8efc7b2b42 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:58.882724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1231/2450] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:26:58.887399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ce847-c499-4d61-87be-e8dc3119e2fa admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:58.897461 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ce847-c499-4d61-87be-e8dc3119e2fa admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:58.902244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1d92f-121a-42a5-9edf-0ac73dcd328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:58.902926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1d92f-121a-42a5-9edf-0ac73dcd328b admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:58.903213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1d92f-121a-42a5-9edf-0ac73dcd328b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:58.903475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1d92f-121a-42a5-9edf-0ac73dcd328b admin admin] PATH_INFO: `/v3/projects/ac3523d6158a48b3be1e64aa1157662a/users/7cad3c9a4ab64ed68955cb3148faf99f/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:58.919465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1d92f-121a-42a5-9edf-0ac73dcd328b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3523d6158a48b3be1e64aa1157662a, user_id=7cad3c9a4ab64ed68955cb3148faf99f, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:58.926255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1d92f-121a-42a5-9edf-0ac73dcd328b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:58.994254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1220/2451] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:26:58 2022] PUT /identity/v3/projects/ac3523d6158a48b3be1e64aa1157662a/users/7cad3c9a4ab64ed68955cb3148faf99f/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:59.004537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a59a4-a0fa-4eda-b571-3f4ba8f6c41d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.004819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a59a4-a0fa-4eda-b571-3f4ba8f6c41d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.005087 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a59a4-a0fa-4eda-b571-3f4ba8f6c41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.040957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-b88a59a4-a0fa-4eda-b571-3f4ba8f6c41d None None] MFA Rules not processed for user `7cad3c9a4ab64ed68955cb3148faf99f`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:59.042257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b88a59a4-a0fa-4eda-b571-3f4ba8f6c41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:59.128745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1232/2452] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:59.133293 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1d92f-121a-42a5-9edf-0ac73dcd328b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:59.203064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1d92f-121a-42a5-9edf-0ac73dcd328b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:59.208130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287db15-fcdd-47e4-83cc-e51d52e7fc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:59.209119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287db15-fcdd-47e4-83cc-e51d52e7fc4f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.209636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287db15-fcdd-47e4-83cc-e51d52e7fc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.210043 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287db15-fcdd-47e4-83cc-e51d52e7fc4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.210732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287db15-fcdd-47e4-83cc-e51d52e7fc4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:59.213127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287db15-fcdd-47e4-83cc-e51d52e7fc4f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:59.309753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1221/2453] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:26:59 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:26:59.313896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a59a4-a0fa-4eda-b571-3f4ba8f6c41d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:59.325203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a59a4-a0fa-4eda-b571-3f4ba8f6c41d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:59.386284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2bd8f-6259-42f8-a235-60fb23c73f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:59.387018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2bd8f-6259-42f8-a235-60fb23c73f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.387202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2bd8f-6259-42f8-a235-60fb23c73f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.387472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2bd8f-6259-42f8-a235-60fb23c73f99 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.388105 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2bd8f-6259-42f8-a235-60fb23c73f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:59.390684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2bd8f-6259-42f8-a235-60fb23c73f99 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:59.490998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:59.497833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1233/2454] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:26:59 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:26:59.503223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287db15-fcdd-47e4-83cc-e51d52e7fc4f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:59.514396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287db15-fcdd-47e4-83cc-e51d52e7fc4f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:59.519463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5f34e-d794-40a8-b3b5-e70a9a5dbb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:59.520133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f34e-d794-40a8-b3b5-e70a9a5dbb1e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.520420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f34e-d794-40a8-b3b5-e70a9a5dbb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.520706 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5f34e-d794-40a8-b3b5-e70a9a5dbb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.521332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5f34e-d794-40a8-b3b5-e70a9a5dbb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:59.523069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5f34e-d794-40a8-b3b5-e70a9a5dbb1e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:59.534497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1222/2455] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:26:59.538698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2bd8f-6259-42f8-a235-60fb23c73f99 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:59.550392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2bd8f-6259-42f8-a235-60fb23c73f99 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:59.555089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee5f6a-9ac8-480e-9b60-e5db6275c4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:59.560723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee5f6a-9ac8-480e-9b60-e5db6275c4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.560723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee5f6a-9ac8-480e-9b60-e5db6275c4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.560723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee5f6a-9ac8-480e-9b60-e5db6275c4f3 admin admin] PATH_INFO: `/v3/projects/d46ed81c829e4f11b23091fda9be2cc3/users/aed901c2c4254dd8bedbb25641e30b42/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.572474 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee5f6a-9ac8-480e-9b60-e5db6275c4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46ed81c829e4f11b23091fda9be2cc3, user_id=aed901c2c4254dd8bedbb25641e30b42, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:59.575349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee5f6a-9ac8-480e-9b60-e5db6275c4f3 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:59.662117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1234/2456] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:26:59 2022] PUT /identity/v3/projects/d46ed81c829e4f11b23091fda9be2cc3/users/aed901c2c4254dd8bedbb25641e30b42/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:59.666144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5f34e-d794-40a8-b3b5-e70a9a5dbb1e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:59.677146 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5f34e-d794-40a8-b3b5-e70a9a5dbb1e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:59.733430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2460e94-e267-49ce-9483-2b9b00cb562e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:59.734323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460e94-e267-49ce-9483-2b9b00cb562e admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.734626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460e94-e267-49ce-9483-2b9b00cb562e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.734894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2460e94-e267-49ce-9483-2b9b00cb562e admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.735529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2460e94-e267-49ce-9483-2b9b00cb562e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:59.737121 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2460e94-e267-49ce-9483-2b9b00cb562e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:59.746852 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1223/2457] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:26:59.751046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee5f6a-9ac8-480e-9b60-e5db6275c4f3 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:59.761794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee5f6a-9ac8-480e-9b60-e5db6275c4f3 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:59.766886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:59.767520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.767791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.768088 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] PATH_INFO: `/v3/projects/d46ed81c829e4f11b23091fda9be2cc3/users/aed901c2c4254dd8bedbb25641e30b42/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.769968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46ed81c829e4f11b23091fda9be2cc3, user_id=aed901c2c4254dd8bedbb25641e30b42, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:59.772636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:59.783378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1235/2458] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:26:59 2022] PUT /identity/v3/projects/d46ed81c829e4f11b23091fda9be2cc3/users/aed901c2c4254dd8bedbb25641e30b42/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:26:59.794923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef79a-1c3f-4fb8-9ca2-6e9f0b79745b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.795275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef79a-1c3f-4fb8-9ca2-6e9f0b79745b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.795553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaef79a-1c3f-4fb8-9ca2-6e9f0b79745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.823860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bdaef79a-1c3f-4fb8-9ca2-6e9f0b79745b None None] MFA Rules not processed for user `aed901c2c4254dd8bedbb25641e30b42`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:26:59.825091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bdaef79a-1c3f-4fb8-9ca2-6e9f0b79745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:59.844036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:26:59.892408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1224/2459] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:26:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:26:59.903494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:26:59.905910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:26:59.961807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e24a5c-c8a9-42ec-b15c-dd3d6750fb66 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:26:59.965436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256df7bf-f752-4bd5-a4fb-010ea0de4d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:26:59.966027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256df7bf-f752-4bd5-a4fb-010ea0de4d83 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:26:59.966299 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256df7bf-f752-4bd5-a4fb-010ea0de4d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:26:59.966595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256df7bf-f752-4bd5-a4fb-010ea0de4d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:26:59.976012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256df7bf-f752-4bd5-a4fb-010ea0de4d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:26:59.977420 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256df7bf-f752-4bd5-a4fb-010ea0de4d83 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:26:59.986993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1236/2460] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:26:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:00.203123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaef79a-1c3f-4fb8-9ca2-6e9f0b79745b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:00.212766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaef79a-1c3f-4fb8-9ca2-6e9f0b79745b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:00.216987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce114ff7-2060-4a4f-8d9c-adcac038a8c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:00.218449 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce114ff7-2060-4a4f-8d9c-adcac038a8c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:00.218754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce114ff7-2060-4a4f-8d9c-adcac038a8c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:00.219197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce114ff7-2060-4a4f-8d9c-adcac038a8c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:00.279473 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce114ff7-2060-4a4f-8d9c-adcac038a8c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:00.281752 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce114ff7-2060-4a4f-8d9c-adcac038a8c5 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:00.291025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1225/2461] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:09.983095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=42,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:10.286894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=48 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:11.686817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256df7bf-f752-4bd5-a4fb-010ea0de4d83 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:11.748492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256df7bf-f752-4bd5-a4fb-010ea0de4d83 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:11.753880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:11.754524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:11.754800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:11.755065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] PATH_INFO: `/v3/users/7cad3c9a4ab64ed68955cb3148faf99f` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:11.756171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cad3c9a4ab64ed68955cb3148faf99f)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:11.758167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:11.891637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cad3c9a4ab64ed68955cb3148faf99f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:11.892284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cad3c9a4ab64ed68955cb3148faf99f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:11.958265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cad3c9a4ab64ed68955cb3148faf99f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:11.970383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cad3c9a4ab64ed68955cb3148faf99f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:11.977285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cad3c9a4ab64ed68955cb3148faf99f'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:11.983220 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1237/2462] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:27:11 2022] DELETE /identity/v3/users/7cad3c9a4ab64ed68955cb3148faf99f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:11.988487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f1556-a541-4529-bf80-9c482060783b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:11.988956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f1556-a541-4529-bf80-9c482060783b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:11.989065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f1556-a541-4529-bf80-9c482060783b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:12.008923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8c9f1556-a541-4529-bf80-9c482060783b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:12.010124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8c9f1556-a541-4529-bf80-9c482060783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:12.069995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1226/2463] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:12.079303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:12.080905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:12.138584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001b320-6a8e-47d0-8b88-60bab94fd22b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:12.143515 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6c11b-ea75-49ca-a9da-4efbee42d10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:12.144123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c11b-ea75-49ca-a9da-4efbee42d10f service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:12.144484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c11b-ea75-49ca-a9da-4efbee42d10f service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:12.144765 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c11b-ea75-49ca-a9da-4efbee42d10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:12.153763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6c11b-ea75-49ca-a9da-4efbee42d10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:12.155211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6c11b-ea75-49ca-a9da-4efbee42d10f service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:12.165376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1238/2464] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:12.548679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f1556-a541-4529-bf80-9c482060783b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:12.550249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8c9f1556-a541-4529-bf80-9c482060783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:12.563387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f1556-a541-4529-bf80-9c482060783b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:12.568343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:12.568998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:12.569274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:12.569540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] PATH_INFO: `/v3/projects/ac3523d6158a48b3be1e64aa1157662a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:12.570789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3523d6158a48b3be1e64aa1157662a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:12.572695 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:12.662375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3523d6158a48b3be1e64aa1157662a', 'request_id': 'req-e3a94810-3ba5-4c03-837e-2db4b553388f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:12.712545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3523d6158a48b3be1e64aa1157662a', 'request_id': 'req-e3a94810-3ba5-4c03-837e-2db4b553388f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:12.718268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3523d6158a48b3be1e64aa1157662a', 'request_id': 'req-e3a94810-3ba5-4c03-837e-2db4b553388f'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:12.730874 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] The token cache is being invalidate because project ac3523d6158a48b3be1e64aa1157662a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:27:12.731164 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:12.731837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:12.733355 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1227/2465] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:27:12 2022] DELETE /identity/v3/projects/ac3523d6158a48b3be1e64aa1157662a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:12.738438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6c11b-ea75-49ca-a9da-4efbee42d10f service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:12.748175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6c11b-ea75-49ca-a9da-4efbee42d10f service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:12.766724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:12.805237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:12.805892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:12.806171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:12.806479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] PATH_INFO: `/v3/users/aed901c2c4254dd8bedbb25641e30b42` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:12.807595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed901c2c4254dd8bedbb25641e30b42)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:12.809507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:12.965431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed901c2c4254dd8bedbb25641e30b42'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:12.966073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed901c2c4254dd8bedbb25641e30b42'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.044009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed901c2c4254dd8bedbb25641e30b42'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.057258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed901c2c4254dd8bedbb25641e30b42'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.064401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f8139c5f-9892-458f-bcb1-d17222700da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed901c2c4254dd8bedbb25641e30b42'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.070199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1239/2466] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:27:12 2022] DELETE /identity/v3/users/aed901c2c4254dd8bedbb25641e30b42 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:13.463254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:13.464766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:13.479453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a94810-3ba5-4c03-837e-2db4b553388f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:13.539016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:13.539639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:13.539915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:13.540184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] PATH_INFO: `/v3/projects/d46ed81c829e4f11b23091fda9be2cc3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:13.541340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d46ed81c829e4f11b23091fda9be2cc3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:13.543319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:13.651339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:13.667844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd46ed81c829e4f11b23091fda9be2cc3', 'request_id': 'req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.729669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd46ed81c829e4f11b23091fda9be2cc3', 'request_id': 'req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.736830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd46ed81c829e4f11b23091fda9be2cc3', 'request_id': 'req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.748697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] The token cache is being invalidate because project d46ed81c829e4f11b23091fda9be2cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:27:13.748982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.749628 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-135f5ea0-f7f8-461c-a75d-f3e629d93fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:13.751433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1228/2467] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:27:13 2022] DELETE /identity/v3/projects/d46ed81c829e4f11b23091fda9be2cc3 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:13.756297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bd54-226e-4976-a212-77c8384656c5 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:13.756811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bd54-226e-4976-a212-77c8384656c5 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:13.756811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7bd54-226e-4976-a212-77c8384656c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:13.777570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-31a7bd54-226e-4976-a212-77c8384656c5 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:13.778818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-31a7bd54-226e-4976-a212-77c8384656c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:13.837438 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1240/2468] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:13.845242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d904db2-aa4b-46cb-b821-7726e1d24d3d None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:13.845523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d904db2-aa4b-46cb-b821-7726e1d24d3d None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:13.845793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d904db2-aa4b-46cb-b821-7726e1d24d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:13.865834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2d904db2-aa4b-46cb-b821-7726e1d24d3d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:13.867064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2d904db2-aa4b-46cb-b821-7726e1d24d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:13.874918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1229/2469] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:13.879828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7bd54-226e-4976-a212-77c8384656c5 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:13.889460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7bd54-226e-4976-a212-77c8384656c5 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:13.894405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:13.894998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:13.895272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:13.895564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:13.896198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:27:13.896494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:13.898077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:13.909782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1241/2470] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:27:13.914086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d904db2-aa4b-46cb-b821-7726e1d24d3d None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:13.923324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d904db2-aa4b-46cb-b821-7726e1d24d3d None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:13.928525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21cfa6-3848-4dcb-a2da-e3b5725daf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:13.929107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21cfa6-3848-4dcb-a2da-e3b5725daf57 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:13.929382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21cfa6-3848-4dcb-a2da-e3b5725daf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:13.929648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21cfa6-3848-4dcb-a2da-e3b5725daf57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:13.930275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21cfa6-3848-4dcb-a2da-e3b5725daf57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:13.932192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21cfa6-3848-4dcb-a2da-e3b5725daf57 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:14.033388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1230/2471] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:27:13 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:14.037597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:14.047840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf42b75-7851-4ce4-8193-7a38363b04f9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:14.074827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:14.106139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dba8c-7a83-4f62-8b91-da45d1aec2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:14.106885 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dba8c-7a83-4f62-8b91-da45d1aec2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.107157 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dba8c-7a83-4f62-8b91-da45d1aec2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.107428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517dba8c-7a83-4f62-8b91-da45d1aec2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.108057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dba8c-7a83-4f62-8b91-da45d1aec2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:14.110216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517dba8c-7a83-4f62-8b91-da45d1aec2f1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:14.240389 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1242/2472] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:27:14 2022] POST /identity/v3/users => generated 285 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:14.244222 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21cfa6-3848-4dcb-a2da-e3b5725daf57 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:14.256645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21cfa6-3848-4dcb-a2da-e3b5725daf57 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:14.261389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f4239-0cd5-4408-9d57-20b9fe9a7b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:14.262004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f4239-0cd5-4408-9d57-20b9fe9a7b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.262281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f4239-0cd5-4408-9d57-20b9fe9a7b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.262587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f4239-0cd5-4408-9d57-20b9fe9a7b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.263229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f4239-0cd5-4408-9d57-20b9fe9a7b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:14.264917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f4239-0cd5-4408-9d57-20b9fe9a7b47 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:14.279770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1231/2473] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:14.281069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dba8c-7a83-4f62-8b91-da45d1aec2f1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:14.293263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517dba8c-7a83-4f62-8b91-da45d1aec2f1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:14.298943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960a901-0cbb-4be0-adf0-cf5fabd5022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:14.299265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960a901-0cbb-4be0-adf0-cf5fabd5022b admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.299569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960a901-0cbb-4be0-adf0-cf5fabd5022b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.299908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960a901-0cbb-4be0-adf0-cf5fabd5022b admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.302472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960a901-0cbb-4be0-adf0-cf5fabd5022b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:14.302472 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960a901-0cbb-4be0-adf0-cf5fabd5022b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:14.312658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1243/2474] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:14.318191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f4239-0cd5-4408-9d57-20b9fe9a7b47 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:14.328446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f4239-0cd5-4408-9d57-20b9fe9a7b47 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:14.333938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c85855-9c6d-4cfa-9e2a-51ab168b8235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:14.334618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85855-9c6d-4cfa-9e2a-51ab168b8235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.334896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85855-9c6d-4cfa-9e2a-51ab168b8235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.335163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c85855-9c6d-4cfa-9e2a-51ab168b8235 admin admin] PATH_INFO: `/v3/projects/8f999025e9ee416aaa208395a5a08e6b/users/e09e6aabff774555811744306651b647/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.349699 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c85855-9c6d-4cfa-9e2a-51ab168b8235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f999025e9ee416aaa208395a5a08e6b, user_id=e09e6aabff774555811744306651b647, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:14.352446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c85855-9c6d-4cfa-9e2a-51ab168b8235 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:14.404238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1232/2475] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:14 2022] PUT /identity/v3/projects/8f999025e9ee416aaa208395a5a08e6b/users/e09e6aabff774555811744306651b647/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:14.416627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623daad-bf5c-45ce-91b0-194b77858bbf None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.417080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623daad-bf5c-45ce-91b0-194b77858bbf None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.417561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623daad-bf5c-45ce-91b0-194b77858bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.452113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-b623daad-bf5c-45ce-91b0-194b77858bbf None None] MFA Rules not processed for user `e09e6aabff774555811744306651b647`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:14.454319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b623daad-bf5c-45ce-91b0-194b77858bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:14.528524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1244/2476] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:14.533114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c85855-9c6d-4cfa-9e2a-51ab168b8235 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:14.609434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c85855-9c6d-4cfa-9e2a-51ab168b8235 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:14.615579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a05f51-e3e8-47e6-8cb9-497cb6def473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:14.616272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a05f51-e3e8-47e6-8cb9-497cb6def473 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.616617 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a05f51-e3e8-47e6-8cb9-497cb6def473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.616896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a05f51-e3e8-47e6-8cb9-497cb6def473 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.617526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a05f51-e3e8-47e6-8cb9-497cb6def473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:14.619466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a05f51-e3e8-47e6-8cb9-497cb6def473 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:14.686359 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1233/2477] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:27:14 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:14.690643 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623daad-bf5c-45ce-91b0-194b77858bbf None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:14.704272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623daad-bf5c-45ce-91b0-194b77858bbf None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:14.774243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3dfea-6511-4e1a-a700-9154b5e5506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:14.774915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3dfea-6511-4e1a-a700-9154b5e5506a admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.775188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3dfea-6511-4e1a-a700-9154b5e5506a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.775477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3dfea-6511-4e1a-a700-9154b5e5506a admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.776249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3dfea-6511-4e1a-a700-9154b5e5506a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:14.778221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3dfea-6511-4e1a-a700-9154b5e5506a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:14.873386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1245/2478] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:27:14 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:27:14.877576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a05f51-e3e8-47e6-8cb9-497cb6def473 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:14.888934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a05f51-e3e8-47e6-8cb9-497cb6def473 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:14.893758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80d910-4c40-4299-911e-84ca45227733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:14.894394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80d910-4c40-4299-911e-84ca45227733 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.894671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80d910-4c40-4299-911e-84ca45227733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.894940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80d910-4c40-4299-911e-84ca45227733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.895606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80d910-4c40-4299-911e-84ca45227733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:14.897159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80d910-4c40-4299-911e-84ca45227733 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:14.909320 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1234/2479] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:14.914176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3dfea-6511-4e1a-a700-9154b5e5506a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:14.924433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3dfea-6511-4e1a-a700-9154b5e5506a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:14.931008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af852e0a-7be7-4251-9ab9-33a5f1a14a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:14.931669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852e0a-7be7-4251-9ab9-33a5f1a14a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:14.931948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852e0a-7be7-4251-9ab9-33a5f1a14a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:14.932437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852e0a-7be7-4251-9ab9-33a5f1a14a01 admin admin] PATH_INFO: `/v3/projects/54a0a37c14ae448e9e0a344eb790e99d/users/85b77ea48e1544358d78f47b9745554e/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:14.949890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af852e0a-7be7-4251-9ab9-33a5f1a14a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a0a37c14ae448e9e0a344eb790e99d, user_id=85b77ea48e1544358d78f47b9745554e, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:14.952601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af852e0a-7be7-4251-9ab9-33a5f1a14a01 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:15.034423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1246/2480] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:14 2022] PUT /identity/v3/projects/54a0a37c14ae448e9e0a344eb790e99d/users/85b77ea48e1544358d78f47b9745554e/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:15.038681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80d910-4c40-4299-911e-84ca45227733 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:15.049027 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80d910-4c40-4299-911e-84ca45227733 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:15.069311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:15.106070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af554c-6621-48ad-8673-c06e629e12a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:15.106655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af554c-6621-48ad-8673-c06e629e12a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:15.106949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af554c-6621-48ad-8673-c06e629e12a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:15.107227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af554c-6621-48ad-8673-c06e629e12a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:15.107814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af554c-6621-48ad-8673-c06e629e12a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:15.109678 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af554c-6621-48ad-8673-c06e629e12a7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:15.120802 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1235/2481] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:15.125012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af852e0a-7be7-4251-9ab9-33a5f1a14a01 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:15.130837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:15.135907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af852e0a-7be7-4251-9ab9-33a5f1a14a01 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:15.140473 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:15.141115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:15.141387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:15.141654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] PATH_INFO: `/v3/projects/54a0a37c14ae448e9e0a344eb790e99d/users/85b77ea48e1544358d78f47b9745554e/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:15.143798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a0a37c14ae448e9e0a344eb790e99d, user_id=85b77ea48e1544358d78f47b9745554e, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:15.146830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:15.157196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1247/2482] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:15 2022] PUT /identity/v3/projects/54a0a37c14ae448e9e0a344eb790e99d/users/85b77ea48e1544358d78f47b9745554e/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:15.168037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bae1-1701-4005-bae3-181a67d036ce None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:15.168403 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bae1-1701-4005-bae3-181a67d036ce None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:15.168653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bae1-1701-4005-bae3-181a67d036ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:15.197780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2077bae1-1701-4005-bae3-181a67d036ce None None] MFA Rules not processed for user `85b77ea48e1544358d78f47b9745554e`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:15.199016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2077bae1-1701-4005-bae3-181a67d036ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:15.266511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1236/2483] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:15.277196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:15.278924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:15.335924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84310056-e7b4-47a8-90b1-cdcf9c56fe0d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:15.340003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873636ba-3a84-46a7-99f0-e1c60882c169 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:15.341486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873636ba-3a84-46a7-99f0-e1c60882c169 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:15.342051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873636ba-3a84-46a7-99f0-e1c60882c169 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:15.342691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873636ba-3a84-46a7-99f0-e1c60882c169 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:15.355977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873636ba-3a84-46a7-99f0-e1c60882c169 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:15.357418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873636ba-3a84-46a7-99f0-e1c60882c169 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:15.368018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1248/2484] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:17.840157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077bae1-1701-4005-bae3-181a67d036ce None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:17.902611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077bae1-1701-4005-bae3-181a67d036ce None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:17.907119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:17.907742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:17.908013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:17.908278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] PATH_INFO: `/v3/users/e09e6aabff774555811744306651b647` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:17.909380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09e6aabff774555811744306651b647)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:17.912057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:18.140101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09e6aabff774555811744306651b647'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:18.142035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:18.187077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09e6aabff774555811744306651b647'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:18.187705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09e6aabff774555811744306651b647'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:18.199560 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09e6aabff774555811744306651b647'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:18.212690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09e6aabff774555811744306651b647'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:18.218973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1237/2485] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:27:17 2022] DELETE /identity/v3/users/e09e6aabff774555811744306651b647 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:18.224043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367fd52-6565-4b26-a089-2e9c6e044bd0 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:18.224521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367fd52-6565-4b26-a089-2e9c6e044bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:18.224634 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367fd52-6565-4b26-a089-2e9c6e044bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:18.243798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f367fd52-6565-4b26-a089-2e9c6e044bd0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:18.245035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f367fd52-6565-4b26-a089-2e9c6e044bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:18.305204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1249/2486] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:18.314638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:18.316649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:18.372413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7b68-a701-4ed5-801a-7a6716dfee42 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:18.375952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40061a5-62d5-48a5-9309-bf0b3c1d41ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:18.376543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40061a5-62d5-48a5-9309-bf0b3c1d41ca service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:18.376814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40061a5-62d5-48a5-9309-bf0b3c1d41ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:18.377081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40061a5-62d5-48a5-9309-bf0b3c1d41ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:18.386732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40061a5-62d5-48a5-9309-bf0b3c1d41ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:18.388141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40061a5-62d5-48a5-9309-bf0b3c1d41ca service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:18.397536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1238/2487] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:18.828563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367fd52-6565-4b26-a089-2e9c6e044bd0 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:18.829996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f367fd52-6565-4b26-a089-2e9c6e044bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:18.843090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367fd52-6565-4b26-a089-2e9c6e044bd0 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:18.848467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:18.849097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:18.849490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:18.849828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] PATH_INFO: `/v3/projects/8f999025e9ee416aaa208395a5a08e6b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:18.851470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f999025e9ee416aaa208395a5a08e6b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:18.852837 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:18.953410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f999025e9ee416aaa208395a5a08e6b', 'request_id': 'req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.010099 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f999025e9ee416aaa208395a5a08e6b', 'request_id': 'req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.017956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f999025e9ee416aaa208395a5a08e6b', 'request_id': 'req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.031452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] The token cache is being invalidate because project 8f999025e9ee416aaa208395a5a08e6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:27:19.031740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.032410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.033923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1250/2488] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:27:18 2022] DELETE /identity/v3/projects/8f999025e9ee416aaa208395a5a08e6b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:19.038275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40061a5-62d5-48a5-9309-bf0b3c1d41ca service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:19.048069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40061a5-62d5-48a5-9309-bf0b3c1d41ca service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:19.104074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:19.104708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:19.104989 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:19.105364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] PATH_INFO: `/v3/users/85b77ea48e1544358d78f47b9745554e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:19.106433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b77ea48e1544358d78f47b9745554e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:19.108369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:19.263166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b77ea48e1544358d78f47b9745554e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.352395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b77ea48e1544358d78f47b9745554e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.353009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b77ea48e1544358d78f47b9745554e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.360961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b77ea48e1544358d78f47b9745554e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.367149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:19.374727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a01f92ef-3dcc-4b01-82ba-3e7ca786f1b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b77ea48e1544358d78f47b9745554e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.381295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1239/2489] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:27:19 2022] DELETE /identity/v3/users/85b77ea48e1544358d78f47b9745554e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:19.753916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:19.755601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:19.770125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9268fc-7fa5-4703-8d3b-bd3d02480f5c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:19.783607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:19.830349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:19.831058 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:19.831376 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:19.832125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] PATH_INFO: `/v3/projects/54a0a37c14ae448e9e0a344eb790e99d` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:19.833636 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a0a37c14ae448e9e0a344eb790e99d)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:19.835570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:19.929908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a0a37c14ae448e9e0a344eb790e99d', 'request_id': 'req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.992350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a0a37c14ae448e9e0a344eb790e99d', 'request_id': 'req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:19.998066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a0a37c14ae448e9e0a344eb790e99d', 'request_id': 'req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:20.009302 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] The token cache is being invalidate because project 54a0a37c14ae448e9e0a344eb790e99d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:27:20.009590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:20.010244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-95d5eeb3-7760-4fe5-89e9-1aa1c30b18c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:20.011752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1251/2490] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:27:19 2022] DELETE /identity/v3/projects/54a0a37c14ae448e9e0a344eb790e99d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:27:20.016820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d75d-7b5a-40e3-9770-bad47ea35354 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.017108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d75d-7b5a-40e3-9770-bad47ea35354 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.017386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d75d-7b5a-40e3-9770-bad47ea35354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.038027 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c921d75d-7b5a-40e3-9770-bad47ea35354 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:20.039267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c921d75d-7b5a-40e3-9770-bad47ea35354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:20.098756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1240/2491] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:20.106823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addccc3-7b2f-40cc-a819-99378aa80eb8 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.107113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addccc3-7b2f-40cc-a819-99378aa80eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.107385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addccc3-7b2f-40cc-a819-99378aa80eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.127450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0addccc3-7b2f-40cc-a819-99378aa80eb8 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:20.128774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0addccc3-7b2f-40cc-a819-99378aa80eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:20.137029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1252/2492] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:27:20.142608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d75d-7b5a-40e3-9770-bad47ea35354 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:20.153016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d75d-7b5a-40e3-9770-bad47ea35354 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:20.157570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:20.158210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.158472 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.158745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.159380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:27:20.160306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:20.161959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:20.173448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1241/2493] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:27:20.184550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addccc3-7b2f-40cc-a819-99378aa80eb8 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:20.194409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addccc3-7b2f-40cc-a819-99378aa80eb8 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:20.199126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c527f7-b1ca-4df3-88a4-cf397168692e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:20.199715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c527f7-b1ca-4df3-88a4-cf397168692e admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.199992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c527f7-b1ca-4df3-88a4-cf397168692e admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.200258 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c527f7-b1ca-4df3-88a4-cf397168692e admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.200878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c527f7-b1ca-4df3-88a4-cf397168692e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:20.202836 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c527f7-b1ca-4df3-88a4-cf397168692e admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:20.275024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1253/2494] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:27:20 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:27:20.279421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:20.289977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053b04d-d750-4633-9207-7c02087801b3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:20.347247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae24d3-cea5-4055-b77c-f748c012a0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:20.348051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae24d3-cea5-4055-b77c-f748c012a0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.348329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae24d3-cea5-4055-b77c-f748c012a0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.348597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae24d3-cea5-4055-b77c-f748c012a0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.349269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae24d3-cea5-4055-b77c-f748c012a0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:20.351143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae24d3-cea5-4055-b77c-f748c012a0b7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:20.419754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1242/2495] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:27:20 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:20.420925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c527f7-b1ca-4df3-88a4-cf397168692e admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:20.431763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c527f7-b1ca-4df3-88a4-cf397168692e admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:20.436414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeba01-6e96-4aaf-b600-ee6a4669a9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:20.437008 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeba01-6e96-4aaf-b600-ee6a4669a9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.437285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeba01-6e96-4aaf-b600-ee6a4669a9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.437554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeba01-6e96-4aaf-b600-ee6a4669a9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.438185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeba01-6e96-4aaf-b600-ee6a4669a9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:20.439799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeba01-6e96-4aaf-b600-ee6a4669a9a9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:20.450287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1254/2496] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:20.454325 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae24d3-cea5-4055-b77c-f748c012a0b7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:20.464984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae24d3-cea5-4055-b77c-f748c012a0b7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:20.469627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750271cc-2d66-43ac-8f00-e7cf5432b425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:20.470212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750271cc-2d66-43ac-8f00-e7cf5432b425 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.470511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750271cc-2d66-43ac-8f00-e7cf5432b425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.470780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750271cc-2d66-43ac-8f00-e7cf5432b425 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.471412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750271cc-2d66-43ac-8f00-e7cf5432b425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:20.473213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750271cc-2d66-43ac-8f00-e7cf5432b425 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:20.487405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1243/2497] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:20.491510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeba01-6e96-4aaf-b600-ee6a4669a9a9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:20.501573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeba01-6e96-4aaf-b600-ee6a4669a9a9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:20.507455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227e650-0ad7-4d7d-a236-f97af6704c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:20.508143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227e650-0ad7-4d7d-a236-f97af6704c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.508424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227e650-0ad7-4d7d-a236-f97af6704c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.508699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227e650-0ad7-4d7d-a236-f97af6704c41 admin admin] PATH_INFO: `/v3/projects/211e8d74cc73473eb37ebf6ba376fa36/users/14d8df23743343cc9f89c1810b121ca3/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.524939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227e650-0ad7-4d7d-a236-f97af6704c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=211e8d74cc73473eb37ebf6ba376fa36, user_id=14d8df23743343cc9f89c1810b121ca3, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:20.527838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227e650-0ad7-4d7d-a236-f97af6704c41 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:20.633738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1255/2498] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:20 2022] PUT /identity/v3/projects/211e8d74cc73473eb37ebf6ba376fa36/users/14d8df23743343cc9f89c1810b121ca3/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:20.645271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b7c0-9ee2-4292-851e-bcc7cbfce868 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.645548 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b7c0-9ee2-4292-851e-bcc7cbfce868 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.645817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b7c0-9ee2-4292-851e-bcc7cbfce868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.673400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-e7c4b7c0-9ee2-4292-851e-bcc7cbfce868 None None] MFA Rules not processed for user `14d8df23743343cc9f89c1810b121ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:20.674631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-e7c4b7c0-9ee2-4292-851e-bcc7cbfce868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:20.722917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:20.745887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1244/2499] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:20.750448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227e650-0ad7-4d7d-a236-f97af6704c41 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:20.803592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:20.814676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227e650-0ad7-4d7d-a236-f97af6704c41 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:20.820675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401da3e-83c8-42c1-8609-b34d5050c48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:20.825833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401da3e-83c8-42c1-8609-b34d5050c48a admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.825833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401da3e-83c8-42c1-8609-b34d5050c48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.825833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0401da3e-83c8-42c1-8609-b34d5050c48a admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.825833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401da3e-83c8-42c1-8609-b34d5050c48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:20.825833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0401da3e-83c8-42c1-8609-b34d5050c48a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:20.914213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1256/2500] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:27:20 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:20.918319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4b7c0-9ee2-4292-851e-bcc7cbfce868 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:20.929667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4b7c0-9ee2-4292-851e-bcc7cbfce868 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:20.986227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b6c91-8a64-49f9-85fc-4c9d27dbd6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:20.986854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b6c91-8a64-49f9-85fc-4c9d27dbd6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:20.987132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b6c91-8a64-49f9-85fc-4c9d27dbd6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:20.987398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b6c91-8a64-49f9-85fc-4c9d27dbd6b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:20.988794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b6c91-8a64-49f9-85fc-4c9d27dbd6b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:20.990775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b6c91-8a64-49f9-85fc-4c9d27dbd6b4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:21.056483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1245/2501] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:27:20 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:21.062367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401da3e-83c8-42c1-8609-b34d5050c48a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:21.074121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0401da3e-83c8-42c1-8609-b34d5050c48a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:21.078841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac8f8-55cd-4cf0-9115-367f1430a01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:21.079427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac8f8-55cd-4cf0-9115-367f1430a01b admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:21.079711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac8f8-55cd-4cf0-9115-367f1430a01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:21.079975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bac8f8-55cd-4cf0-9115-367f1430a01b admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:21.080597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bac8f8-55cd-4cf0-9115-367f1430a01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:21.082154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bac8f8-55cd-4cf0-9115-367f1430a01b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:21.091802 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1257/2502] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:21.095992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b6c91-8a64-49f9-85fc-4c9d27dbd6b4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:21.106949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b6c91-8a64-49f9-85fc-4c9d27dbd6b4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:21.111831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bcc7a-8dcc-4aea-a8d9-4274f14fa184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:21.112487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bcc7a-8dcc-4aea-a8d9-4274f14fa184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:21.113576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bcc7a-8dcc-4aea-a8d9-4274f14fa184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:21.114002 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bcc7a-8dcc-4aea-a8d9-4274f14fa184 admin admin] PATH_INFO: `/v3/projects/5ece42619f704cf8b118c01804715bf2/users/3b0c128c8b4d41fface21e8736ea58ed/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:21.128424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bcc7a-8dcc-4aea-a8d9-4274f14fa184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ece42619f704cf8b118c01804715bf2, user_id=3b0c128c8b4d41fface21e8736ea58ed, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:21.131432 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bcc7a-8dcc-4aea-a8d9-4274f14fa184 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:21.204148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1246/2503] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:21 2022] PUT /identity/v3/projects/5ece42619f704cf8b118c01804715bf2/users/3b0c128c8b4d41fface21e8736ea58ed/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:21.208244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac8f8-55cd-4cf0-9115-367f1430a01b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:21.218264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bac8f8-55cd-4cf0-9115-367f1430a01b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:21.274916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86827fa5-8e26-4ea2-a000-c31d3292fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:21.275512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827fa5-8e26-4ea2-a000-c31d3292fdab admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:21.275791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827fa5-8e26-4ea2-a000-c31d3292fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:21.276093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86827fa5-8e26-4ea2-a000-c31d3292fdab admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:21.276724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86827fa5-8e26-4ea2-a000-c31d3292fdab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:21.278305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86827fa5-8e26-4ea2-a000-c31d3292fdab admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:21.288473 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1258/2504] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:21.292619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bcc7a-8dcc-4aea-a8d9-4274f14fa184 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:21.303566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bcc7a-8dcc-4aea-a8d9-4274f14fa184 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:21.308098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:21.308730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:21.309005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:21.309271 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] PATH_INFO: `/v3/projects/5ece42619f704cf8b118c01804715bf2/users/3b0c128c8b4d41fface21e8736ea58ed/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:21.311216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ece42619f704cf8b118c01804715bf2, user_id=3b0c128c8b4d41fface21e8736ea58ed, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:21.313847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:21.323851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1247/2505] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:21 2022] PUT /identity/v3/projects/5ece42619f704cf8b118c01804715bf2/users/3b0c128c8b4d41fface21e8736ea58ed/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:21.334369 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d806c-9766-4d03-b8e9-817b0f4d5eb0 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:21.334696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d806c-9766-4d03-b8e9-817b0f4d5eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:21.335000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d806c-9766-4d03-b8e9-817b0f4d5eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:21.363906 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5b9d806c-9766-4d03-b8e9-817b0f4d5eb0 None None] MFA Rules not processed for user `3b0c128c8b4d41fface21e8736ea58ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:21.365113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5b9d806c-9766-4d03-b8e9-817b0f4d5eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:21.432585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1259/2506] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:21.443504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:21.445164 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:21.501708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad189f-3821-4c24-be7e-f8f995bdf0f5 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:21.505856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8c292-1d6d-49dd-b40e-3eb5d32d0cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:21.506481 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8c292-1d6d-49dd-b40e-3eb5d32d0cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:21.506753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8c292-1d6d-49dd-b40e-3eb5d32d0cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:21.507018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8c292-1d6d-49dd-b40e-3eb5d32d0cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:21.554017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:21.564542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8c292-1d6d-49dd-b40e-3eb5d32d0cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:21.566759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8c292-1d6d-49dd-b40e-3eb5d32d0cd5 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:21.575578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1248/2507] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:27.819692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d806c-9766-4d03-b8e9-817b0f4d5eb0 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:27.821139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5b9d806c-9766-4d03-b8e9-817b0f4d5eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:27.833979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d806c-9766-4d03-b8e9-817b0f4d5eb0 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:27.837597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c70ef-0eb4-46a1-9044-daed4308d9dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:27.838188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c70ef-0eb4-46a1-9044-daed4308d9dc service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:27.838505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c70ef-0eb4-46a1-9044-daed4308d9dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:27.838777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c70ef-0eb4-46a1-9044-daed4308d9dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:27.847781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c70ef-0eb4-46a1-9044-daed4308d9dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:27.849193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c70ef-0eb4-46a1-9044-daed4308d9dc service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:27.858807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1260/2508] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:31.614892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:37.854442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:54.880756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8c292-1d6d-49dd-b40e-3eb5d32d0cd5 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:54.942131 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8c292-1d6d-49dd-b40e-3eb5d32d0cd5 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:54.946747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:54.947368 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:54.947646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:54.947919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] PATH_INFO: `/v3/users/14d8df23743343cc9f89c1810b121ca3` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:54.949046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d8df23743343cc9f89c1810b121ca3)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:54.951018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:55.115249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d8df23743343cc9f89c1810b121ca3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:55.226059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d8df23743343cc9f89c1810b121ca3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:55.226674 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d8df23743343cc9f89c1810b121ca3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:55.234300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d8df23743343cc9f89c1810b121ca3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:55.247153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d8df23743343cc9f89c1810b121ca3'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:55.253166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1249/2509] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:27:54 2022] DELETE /identity/v3/users/14d8df23743343cc9f89c1810b121ca3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:55.258588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b3cb4-b6c0-482a-941a-ab9d66ae005e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:55.259071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b3cb4-b6c0-482a-941a-ab9d66ae005e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:55.259172 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b3cb4-b6c0-482a-941a-ab9d66ae005e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:55.277960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5f2b3cb4-b6c0-482a-941a-ab9d66ae005e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:55.279188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5f2b3cb4-b6c0-482a-941a-ab9d66ae005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:55.338043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1261/2510] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:55.352455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:55.352455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:55.414866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbcaeb9-a792-4eb6-9d1b-afa3bd20ac9d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:55.417583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a0c60-6fd5-4999-a65e-3f1697d16545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:55.418787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a0c60-6fd5-4999-a65e-3f1697d16545 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:55.418787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a0c60-6fd5-4999-a65e-3f1697d16545 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:55.418951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a0c60-6fd5-4999-a65e-3f1697d16545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:55.429866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a0c60-6fd5-4999-a65e-3f1697d16545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:55.431316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a0c60-6fd5-4999-a65e-3f1697d16545 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:55.440629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1250/2511] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:55.834506 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b3cb4-b6c0-482a-941a-ab9d66ae005e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:55.835933 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5f2b3cb4-b6c0-482a-941a-ab9d66ae005e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:55.852645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b3cb4-b6c0-482a-941a-ab9d66ae005e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:55.853937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:55.854766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:55.855038 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:55.855303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] PATH_INFO: `/v3/projects/211e8d74cc73473eb37ebf6ba376fa36` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:55.856417 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=211e8d74cc73473eb37ebf6ba376fa36)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:55.859268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:55.966006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '211e8d74cc73473eb37ebf6ba376fa36', 'request_id': 'req-16d7a325-21c8-4dec-8d54-69afbfc2aa94'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.048447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '211e8d74cc73473eb37ebf6ba376fa36', 'request_id': 'req-16d7a325-21c8-4dec-8d54-69afbfc2aa94'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.054815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '211e8d74cc73473eb37ebf6ba376fa36', 'request_id': 'req-16d7a325-21c8-4dec-8d54-69afbfc2aa94'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.066136 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] The token cache is being invalidate because project 211e8d74cc73473eb37ebf6ba376fa36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:27:56.066498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.067187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.068649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1262/2512] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:27:55 2022] DELETE /identity/v3/projects/211e8d74cc73473eb37ebf6ba376fa36 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:56.073213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a0c60-6fd5-4999-a65e-3f1697d16545 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:56.083832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a0c60-6fd5-4999-a65e-3f1697d16545 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:56.102778 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:56.142230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:56.142964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:56.143241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:56.143572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] PATH_INFO: `/v3/users/3b0c128c8b4d41fface21e8736ea58ed` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:56.144847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b0c128c8b4d41fface21e8736ea58ed)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:56.146804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:56.296790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b0c128c8b4d41fface21e8736ea58ed'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.362661 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b0c128c8b4d41fface21e8736ea58ed'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.364099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b0c128c8b4d41fface21e8736ea58ed'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.375341 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b0c128c8b4d41fface21e8736ea58ed'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.387528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b0c128c8b4d41fface21e8736ea58ed'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:56.393576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1251/2513] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:27:56 2022] DELETE /identity/v3/users/3b0c128c8b4d41fface21e8736ea58ed => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:56.736897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:56.738447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:56.805430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:56.806954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:56.812853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7a325-21c8-4dec-8d54-69afbfc2aa94 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:56.834930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:56.878280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dcf9b-f460-4c53-91ed-fc2c64d9a072 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:56.883335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:56.883983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:56.884249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:56.884521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] PATH_INFO: `/v3/projects/5ece42619f704cf8b118c01804715bf2` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:56.885119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d5a0b-5526-41ba-8109-35f98d96ab4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:56.885637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ece42619f704cf8b118c01804715bf2)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:56.885768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d5a0b-5526-41ba-8109-35f98d96ab4c service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:56.885987 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d5a0b-5526-41ba-8109-35f98d96ab4c service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:56.886252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d5a0b-5526-41ba-8109-35f98d96ab4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:56.887573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:56.888021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a78d5a0b-5526-41ba-8109-35f98d96ab4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:56.948932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d5a0b-5526-41ba-8109-35f98d96ab4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:56.950575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d5a0b-5526-41ba-8109-35f98d96ab4c service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:56.960289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1263/2514] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:27:56.983429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:56.987388 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ece42619f704cf8b118c01804715bf2', 'request_id': 'req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:57.048940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ece42619f704cf8b118c01804715bf2', 'request_id': 'req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:57.055823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ece42619f704cf8b118c01804715bf2', 'request_id': 'req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:57.069247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] The token cache is being invalidate because project 5ece42619f704cf8b118c01804715bf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:27:57.069439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:57.070174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d1f72e32-6f5d-46d9-b0ce-0faeefa5c12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:57.071773 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1252/2515] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:27:56 2022] DELETE /identity/v3/projects/5ece42619f704cf8b118c01804715bf2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:27:57.077018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3c0bc-a781-4a7b-85b8-c61612b53ade None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.077377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3c0bc-a781-4a7b-85b8-c61612b53ade None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.077803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3c0bc-a781-4a7b-85b8-c61612b53ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.098422 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-63f3c0bc-a781-4a7b-85b8-c61612b53ade None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:57.099693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-63f3c0bc-a781-4a7b-85b8-c61612b53ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:57.163117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1264/2516] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:57.171040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17222c8-6f83-4beb-be5f-b5bd40ac9cd8 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.171522 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17222c8-6f83-4beb-be5f-b5bd40ac9cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.171619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17222c8-6f83-4beb-be5f-b5bd40ac9cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.192076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c17222c8-6f83-4beb-be5f-b5bd40ac9cd8 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:57.193329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c17222c8-6f83-4beb-be5f-b5bd40ac9cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:57.201482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1253/2517] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:57.205792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3c0bc-a781-4a7b-85b8-c61612b53ade None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:57.215864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3c0bc-a781-4a7b-85b8-c61612b53ade None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:57.220602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:57.221324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.221605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.221872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.222542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:27:57.223458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:57.225039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:57.237749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1265/2518] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:27:57.245987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17222c8-6f83-4beb-be5f-b5bd40ac9cd8 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:57.255377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17222c8-6f83-4beb-be5f-b5bd40ac9cd8 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:57.260074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edc735-094b-44bb-a36a-f3221a84781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:57.260664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edc735-094b-44bb-a36a-f3221a84781f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.260934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edc735-094b-44bb-a36a-f3221a84781f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.261197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edc735-094b-44bb-a36a-f3221a84781f admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.261820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edc735-094b-44bb-a36a-f3221a84781f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:57.263765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edc735-094b-44bb-a36a-f3221a84781f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:57.331738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1254/2519] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:27:57 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:57.336110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:57.346013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7b5aa-77e8-4985-af3d-16566689c67d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:57.389459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:57.402981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d676a-04b5-41d2-b302-6065ec35b73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:57.403583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d676a-04b5-41d2-b302-6065ec35b73f admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.403860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d676a-04b5-41d2-b302-6065ec35b73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.404133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d676a-04b5-41d2-b302-6065ec35b73f admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.404750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d676a-04b5-41d2-b302-6065ec35b73f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:57.406758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d676a-04b5-41d2-b302-6065ec35b73f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:57.534507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1266/2520] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:27:57 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:57.536276 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edc735-094b-44bb-a36a-f3221a84781f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:57.548141 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edc735-094b-44bb-a36a-f3221a84781f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:57.559667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd986d-1147-4e75-83f8-a72dc166dc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:57.560469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd986d-1147-4e75-83f8-a72dc166dc9c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.560746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd986d-1147-4e75-83f8-a72dc166dc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.561065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd986d-1147-4e75-83f8-a72dc166dc9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.561739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd986d-1147-4e75-83f8-a72dc166dc9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:57.563532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd986d-1147-4e75-83f8-a72dc166dc9c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:57.574789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1255/2521] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:27:57.578921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d676a-04b5-41d2-b302-6065ec35b73f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:57.589230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d676a-04b5-41d2-b302-6065ec35b73f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:57.593972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474978c2-6be9-43a4-8813-6e2eb26c5a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:57.594596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474978c2-6be9-43a4-8813-6e2eb26c5a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.594869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474978c2-6be9-43a4-8813-6e2eb26c5a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.595133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474978c2-6be9-43a4-8813-6e2eb26c5a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.595754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474978c2-6be9-43a4-8813-6e2eb26c5a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:57.597400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474978c2-6be9-43a4-8813-6e2eb26c5a5f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:57.608380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1267/2522] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:57.612939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd986d-1147-4e75-83f8-a72dc166dc9c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:57.623770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd986d-1147-4e75-83f8-a72dc166dc9c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:57.628560 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf4341-2030-4d50-acc3-2327c22a0adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:57.629839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf4341-2030-4d50-acc3-2327c22a0adf admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.630163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf4341-2030-4d50-acc3-2327c22a0adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.630500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf4341-2030-4d50-acc3-2327c22a0adf admin admin] PATH_INFO: `/v3/projects/41fd5aed7eca461b8ac6c376fb574d90/users/ad6bb6a1b868497f9b25ee88e3002eb0/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.648022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf4341-2030-4d50-acc3-2327c22a0adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=41fd5aed7eca461b8ac6c376fb574d90, user_id=ad6bb6a1b868497f9b25ee88e3002eb0, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:57.651164 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf4341-2030-4d50-acc3-2327c22a0adf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:57.706697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1256/2523] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:57 2022] PUT /identity/v3/projects/41fd5aed7eca461b8ac6c376fb574d90/users/ad6bb6a1b868497f9b25ee88e3002eb0/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:57.717210 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d515f0-db6d-4ea3-bf3c-64b053845979 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.717490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d515f0-db6d-4ea3-bf3c-64b053845979 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.717757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d515f0-db6d-4ea3-bf3c-64b053845979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.749718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-77d515f0-db6d-4ea3-bf3c-64b053845979 None None] MFA Rules not processed for user `ad6bb6a1b868497f9b25ee88e3002eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:57.751050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-77d515f0-db6d-4ea3-bf3c-64b053845979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:57.814417 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1268/2524] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:57.819618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf4341-2030-4d50-acc3-2327c22a0adf admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:57.883430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf4341-2030-4d50-acc3-2327c22a0adf admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:57.888028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b110b-e70e-4b2f-8c2a-081736458745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:57.888625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b110b-e70e-4b2f-8c2a-081736458745 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:57.888902 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b110b-e70e-4b2f-8c2a-081736458745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:57.889171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b110b-e70e-4b2f-8c2a-081736458745 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:57.889799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b110b-e70e-4b2f-8c2a-081736458745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:57.891756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b110b-e70e-4b2f-8c2a-081736458745 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:57.981640 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1257/2525] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:27:57 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:57.992511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d515f0-db6d-4ea3-bf3c-64b053845979 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:58.003159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d515f0-db6d-4ea3-bf3c-64b053845979 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:58.067214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e911b0-c950-4a84-946a-29ba17583459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:58.067923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e911b0-c950-4a84-946a-29ba17583459 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:58.068203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e911b0-c950-4a84-946a-29ba17583459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:58.068466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e911b0-c950-4a84-946a-29ba17583459 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:58.069091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e911b0-c950-4a84-946a-29ba17583459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:58.071005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e911b0-c950-4a84-946a-29ba17583459 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:58.175829 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1269/2526] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:27:57 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:27:58.180297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b110b-e70e-4b2f-8c2a-081736458745 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:58.191157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b110b-e70e-4b2f-8c2a-081736458745 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:58.195878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900cbcd-2acd-4933-b017-d14c8df9e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:58.196517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900cbcd-2acd-4933-b017-d14c8df9e28d admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:58.196797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900cbcd-2acd-4933-b017-d14c8df9e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:58.197071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900cbcd-2acd-4933-b017-d14c8df9e28d admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:58.197705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900cbcd-2acd-4933-b017-d14c8df9e28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:58.199332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900cbcd-2acd-4933-b017-d14c8df9e28d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:58.210886 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1258/2527] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:27:58.215056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e911b0-c950-4a84-946a-29ba17583459 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:58.225786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e911b0-c950-4a84-946a-29ba17583459 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:58.230356 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a5284-a29d-4c59-b1a0-772e10731332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:58.231009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a5284-a29d-4c59-b1a0-772e10731332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:58.231285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a5284-a29d-4c59-b1a0-772e10731332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:58.231556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a5284-a29d-4c59-b1a0-772e10731332 admin admin] PATH_INFO: `/v3/projects/e6adb154f82b422989eb8e81f7a45c20/users/a7e7870061074a82b1c201e0c23d658b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:58.246578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a5284-a29d-4c59-b1a0-772e10731332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6adb154f82b422989eb8e81f7a45c20, user_id=a7e7870061074a82b1c201e0c23d658b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:58.251323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a5284-a29d-4c59-b1a0-772e10731332 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:58.324587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1270/2528] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:58 2022] PUT /identity/v3/projects/e6adb154f82b422989eb8e81f7a45c20/users/a7e7870061074a82b1c201e0c23d658b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:58.328561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900cbcd-2acd-4933-b017-d14c8df9e28d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:58.339377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900cbcd-2acd-4933-b017-d14c8df9e28d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:58.383458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:58.399812 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719a666-bb52-475c-9e75-421622d97129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:58.400445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719a666-bb52-475c-9e75-421622d97129 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:58.400831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719a666-bb52-475c-9e75-421622d97129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:58.401098 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719a666-bb52-475c-9e75-421622d97129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:58.401723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719a666-bb52-475c-9e75-421622d97129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:58.403332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719a666-bb52-475c-9e75-421622d97129 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:58.416915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1259/2529] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:27:58.418388 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a5284-a29d-4c59-b1a0-772e10731332 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:58.429172 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a5284-a29d-4c59-b1a0-772e10731332 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:58.433597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:58.434230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:58.434531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:58.434797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] PATH_INFO: `/v3/projects/e6adb154f82b422989eb8e81f7a45c20/users/a7e7870061074a82b1c201e0c23d658b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:58.436797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6adb154f82b422989eb8e81f7a45c20, user_id=a7e7870061074a82b1c201e0c23d658b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:58.439436 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:58.450287 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1271/2530] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:27:58 2022] PUT /identity/v3/projects/e6adb154f82b422989eb8e81f7a45c20/users/a7e7870061074a82b1c201e0c23d658b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:58.517138 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48ff8-b1e9-4dcd-a2e5-12bf88762fae None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:58.517366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48ff8-b1e9-4dcd-a2e5-12bf88762fae None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:58.517646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d48ff8-b1e9-4dcd-a2e5-12bf88762fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:58.546177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c9d48ff8-b1e9-4dcd-a2e5-12bf88762fae None None] MFA Rules not processed for user `a7e7870061074a82b1c201e0c23d658b`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:58.547429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c9d48ff8-b1e9-4dcd-a2e5-12bf88762fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:58.622501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1260/2531] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:58.633811 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:58.635533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:58.646138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:58.694188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9de37-0308-4753-a72c-edc8033f36e2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:58.697680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b164081-b62c-4ddf-8dd0-73c4ed064d38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:58.698268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164081-b62c-4ddf-8dd0-73c4ed064d38 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:58.698576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164081-b62c-4ddf-8dd0-73c4ed064d38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:58.698839 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b164081-b62c-4ddf-8dd0-73c4ed064d38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:58.709494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b164081-b62c-4ddf-8dd0-73c4ed064d38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:58.710961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b164081-b62c-4ddf-8dd0-73c4ed064d38 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:58.720896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1272/2532] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:58.788780 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48ff8-b1e9-4dcd-a2e5-12bf88762fae None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:58.851443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d48ff8-b1e9-4dcd-a2e5-12bf88762fae None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:58.856026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:58.856646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:58.856921 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:58.857191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] PATH_INFO: `/v3/users/ad6bb6a1b868497f9b25ee88e3002eb0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:58.858309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad6bb6a1b868497f9b25ee88e3002eb0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:58.860502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:58.995968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad6bb6a1b868497f9b25ee88e3002eb0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.038079 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad6bb6a1b868497f9b25ee88e3002eb0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.038804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad6bb6a1b868497f9b25ee88e3002eb0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.046103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad6bb6a1b868497f9b25ee88e3002eb0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.048105 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:27:59.058761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad6bb6a1b868497f9b25ee88e3002eb0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.064547 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1261/2533] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:27:58 2022] DELETE /identity/v3/users/ad6bb6a1b868497f9b25ee88e3002eb0 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:59.069787 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7efd2-72a3-4737-b293-cb3dcef01696 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:59.074303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7efd2-72a3-4737-b293-cb3dcef01696 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:59.074303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7efd2-72a3-4737-b293-cb3dcef01696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:59.090030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3bf7efd2-72a3-4737-b293-cb3dcef01696 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:27:59.091276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3bf7efd2-72a3-4737-b293-cb3dcef01696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:59.151941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1273/2534] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:27:59.163246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:59.164679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:59.224174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67825231-222b-4691-98cc-a53c34875767 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:59.227832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f8751-c663-4cc6-a9e2-02f6c966f91f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:59.228433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f8751-c663-4cc6-a9e2-02f6c966f91f service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:59.228713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f8751-c663-4cc6-a9e2-02f6c966f91f service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:59.228982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f8751-c663-4cc6-a9e2-02f6c966f91f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:59.238438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f8751-c663-4cc6-a9e2-02f6c966f91f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:59.239922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f8751-c663-4cc6-a9e2-02f6c966f91f service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:59.250181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1262/2535] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:27:59.645884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7efd2-72a3-4737-b293-cb3dcef01696 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:59.647359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3bf7efd2-72a3-4737-b293-cb3dcef01696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:27:59.663274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7efd2-72a3-4737-b293-cb3dcef01696 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:59.679155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:59.679155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:59.679155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:59.679155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] PATH_INFO: `/v3/projects/41fd5aed7eca461b8ac6c376fb574d90` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:59.680676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41fd5aed7eca461b8ac6c376fb574d90)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:59.683265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:27:59.793853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41fd5aed7eca461b8ac6c376fb574d90', 'request_id': 'req-cc28163c-0249-4ccd-96af-b3e1af1b25f6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.865608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41fd5aed7eca461b8ac6c376fb574d90', 'request_id': 'req-cc28163c-0249-4ccd-96af-b3e1af1b25f6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.871796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41fd5aed7eca461b8ac6c376fb574d90', 'request_id': 'req-cc28163c-0249-4ccd-96af-b3e1af1b25f6'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.883277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] The token cache is being invalidate because project 41fd5aed7eca461b8ac6c376fb574d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:27:59.883562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.884183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:27:59.885690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1274/2536] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:27:59 2022] DELETE /identity/v3/projects/41fd5aed7eca461b8ac6c376fb574d90 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:27:59.889615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f8751-c663-4cc6-a9e2-02f6c966f91f service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:27:59.900158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f8751-c663-4cc6-a9e2-02f6c966f91f service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:27:59.956440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:27:59.957997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:27:59.958299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:27:59.958602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] PATH_INFO: `/v3/users/a7e7870061074a82b1c201e0c23d658b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:27:59.959738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e7870061074a82b1c201e0c23d658b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:27:59.961648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:00.148689 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e7870061074a82b1c201e0c23d658b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:00.232028 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e7870061074a82b1c201e0c23d658b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:00.232631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e7870061074a82b1c201e0c23d658b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:00.241402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e7870061074a82b1c201e0c23d658b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:00.255213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-4989d353-70f1-4d15-8fa2-74e6ca371095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e7870061074a82b1c201e0c23d658b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:00.261988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1263/2537] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:27:59 2022] DELETE /identity/v3/users/a7e7870061074a82b1c201e0c23d658b => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:00.938966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:00.940872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:00.954498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28163c-0249-4ccd-96af-b3e1af1b25f6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:01.010295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:01.043192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:01.043834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.044140 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.044412 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] PATH_INFO: `/v3/projects/e6adb154f82b422989eb8e81f7a45c20` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.045551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6adb154f82b422989eb8e81f7a45c20)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:01.047527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:01.181354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6adb154f82b422989eb8e81f7a45c20', 'request_id': 'req-5617ccd8-8773-4e45-88e4-2de18f005bfd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:01.276899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6adb154f82b422989eb8e81f7a45c20', 'request_id': 'req-5617ccd8-8773-4e45-88e4-2de18f005bfd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:01.290445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6adb154f82b422989eb8e81f7a45c20', 'request_id': 'req-5617ccd8-8773-4e45-88e4-2de18f005bfd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:01.302859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] The token cache is being invalidate because project e6adb154f82b422989eb8e81f7a45c20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:28:01.303363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:01.304351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-5617ccd8-8773-4e45-88e4-2de18f005bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:01.306326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1275/2538] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:28:00 2022] DELETE /identity/v3/projects/e6adb154f82b422989eb8e81f7a45c20 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:01.312795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057474c7-a215-4e1e-8d03-88ff2f67795f None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.313323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057474c7-a215-4e1e-8d03-88ff2f67795f None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.313421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057474c7-a215-4e1e-8d03-88ff2f67795f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.322550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:01.334184 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-057474c7-a215-4e1e-8d03-88ff2f67795f None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:01.335439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-057474c7-a215-4e1e-8d03-88ff2f67795f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:01.393872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1264/2539] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:01.402402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637ec7-0f6d-45ec-8a9d-81cb6dedd702 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.402685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637ec7-0f6d-45ec-8a9d-81cb6dedd702 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.402953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95637ec7-0f6d-45ec-8a9d-81cb6dedd702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.423174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-95637ec7-0f6d-45ec-8a9d-81cb6dedd702 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:01.427567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-95637ec7-0f6d-45ec-8a9d-81cb6dedd702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:01.432390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1276/2540] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:01.436786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057474c7-a215-4e1e-8d03-88ff2f67795f None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:01.447266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057474c7-a215-4e1e-8d03-88ff2f67795f None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:01.452607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:01.453202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.453485 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.453791 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.454446 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:28:01.454744 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:01.456338 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:01.468763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1265/2541] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:28:01.473119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95637ec7-0f6d-45ec-8a9d-81cb6dedd702 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:01.482660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95637ec7-0f6d-45ec-8a9d-81cb6dedd702 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:01.487758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41a852-97ac-4665-9f52-e6f8307a7041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:01.488342 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41a852-97ac-4665-9f52-e6f8307a7041 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.488612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41a852-97ac-4665-9f52-e6f8307a7041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.488912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41a852-97ac-4665-9f52-e6f8307a7041 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.489538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41a852-97ac-4665-9f52-e6f8307a7041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:01.491644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41a852-97ac-4665-9f52-e6f8307a7041 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:01.597586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1277/2542] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:28:01 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:01.598831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:01.610672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50626e-6101-4e86-971d-a99783af957b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:01.669013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1944b7-b270-409e-b2e8-c2763843105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:01.669616 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1944b7-b270-409e-b2e8-c2763843105a admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.669891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1944b7-b270-409e-b2e8-c2763843105a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.670160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1944b7-b270-409e-b2e8-c2763843105a admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.670840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1944b7-b270-409e-b2e8-c2763843105a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:01.673330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1944b7-b270-409e-b2e8-c2763843105a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:01.744561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1266/2543] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:28:01 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:28:01.745786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41a852-97ac-4665-9f52-e6f8307a7041 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:01.757264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41a852-97ac-4665-9f52-e6f8307a7041 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:01.761942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0997788-8575-43b4-a8a0-05aae4354153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:01.762576 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0997788-8575-43b4-a8a0-05aae4354153 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.762858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0997788-8575-43b4-a8a0-05aae4354153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.763130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0997788-8575-43b4-a8a0-05aae4354153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.763764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0997788-8575-43b4-a8a0-05aae4354153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:01.765360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0997788-8575-43b4-a8a0-05aae4354153 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:01.775876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1278/2544] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:28:01.780056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1944b7-b270-409e-b2e8-c2763843105a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:01.790540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1944b7-b270-409e-b2e8-c2763843105a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:01.795199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efd331-599e-4333-832b-4ba6465c0b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:01.795787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efd331-599e-4333-832b-4ba6465c0b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.796063 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efd331-599e-4333-832b-4ba6465c0b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.796331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efd331-599e-4333-832b-4ba6465c0b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.798056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efd331-599e-4333-832b-4ba6465c0b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:01.798639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efd331-599e-4333-832b-4ba6465c0b18 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:01.810328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1267/2545] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:01.814365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0997788-8575-43b4-a8a0-05aae4354153 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:01.825357 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0997788-8575-43b4-a8a0-05aae4354153 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:01.829918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c273754-d88f-4dd7-b086-efb0075a91d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:01.830621 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c273754-d88f-4dd7-b086-efb0075a91d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.830870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c273754-d88f-4dd7-b086-efb0075a91d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.831138 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c273754-d88f-4dd7-b086-efb0075a91d1 admin admin] PATH_INFO: `/v3/projects/e4d69be9c90c441cbab25bc41031eeae/users/0836c198b01248bdaa3d0a1854237210/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.845890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c273754-d88f-4dd7-b086-efb0075a91d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d69be9c90c441cbab25bc41031eeae, user_id=0836c198b01248bdaa3d0a1854237210, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:01.848628 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c273754-d88f-4dd7-b086-efb0075a91d1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:01.906249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1279/2546] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:28:01 2022] PUT /identity/v3/projects/e4d69be9c90c441cbab25bc41031eeae/users/0836c198b01248bdaa3d0a1854237210/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:28:01.916623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f688-c467-4671-b35e-f0af8551da33 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:01.916898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f688-c467-4671-b35e-f0af8551da33 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:01.917166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f688-c467-4671-b35e-f0af8551da33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:01.945965 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-a9f3f688-c467-4671-b35e-f0af8551da33 None None] MFA Rules not processed for user `0836c198b01248bdaa3d0a1854237210`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:01.947191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a9f3f688-c467-4671-b35e-f0af8551da33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:02.010135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:02.013919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1268/2547] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:02.018572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c273754-d88f-4dd7-b086-efb0075a91d1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:02.082628 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c273754-d88f-4dd7-b086-efb0075a91d1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:02.088278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bad061-fc6e-4a26-ae46-35ab8a93bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:02.088874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bad061-fc6e-4a26-ae46-35ab8a93bce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:02.089151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bad061-fc6e-4a26-ae46-35ab8a93bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:02.089449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bad061-fc6e-4a26-ae46-35ab8a93bce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:02.090069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bad061-fc6e-4a26-ae46-35ab8a93bce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:02.091993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bad061-fc6e-4a26-ae46-35ab8a93bce3 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:02.096827 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:02.205508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1280/2548] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:28:02 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:02.210445 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3f688-c467-4671-b35e-f0af8551da33 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:02.221190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3f688-c467-4671-b35e-f0af8551da33 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:02.280771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571df14-580c-4fc8-b2d8-748feba2503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:02.281380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571df14-580c-4fc8-b2d8-748feba2503b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:02.281660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571df14-580c-4fc8-b2d8-748feba2503b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:02.281931 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571df14-580c-4fc8-b2d8-748feba2503b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:02.282591 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571df14-580c-4fc8-b2d8-748feba2503b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:02.284507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571df14-580c-4fc8-b2d8-748feba2503b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:02.369227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1269/2549] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:28:02 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:28:02.373440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bad061-fc6e-4a26-ae46-35ab8a93bce3 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:02.391715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bad061-fc6e-4a26-ae46-35ab8a93bce3 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:02.397913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c263e-7ea4-409f-b82d-8ceecab31a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:02.399211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c263e-7ea4-409f-b82d-8ceecab31a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:02.399492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c263e-7ea4-409f-b82d-8ceecab31a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:02.399757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c263e-7ea4-409f-b82d-8ceecab31a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:02.400385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c263e-7ea4-409f-b82d-8ceecab31a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:02.402076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c263e-7ea4-409f-b82d-8ceecab31a55 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:02.413023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1281/2550] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:02.420925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571df14-580c-4fc8-b2d8-748feba2503b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:02.431535 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571df14-580c-4fc8-b2d8-748feba2503b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:02.436362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037702ab-13ec-44ad-83c2-ff621d1246ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:02.437014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037702ab-13ec-44ad-83c2-ff621d1246ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:02.437295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037702ab-13ec-44ad-83c2-ff621d1246ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:02.437570 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037702ab-13ec-44ad-83c2-ff621d1246ce admin admin] PATH_INFO: `/v3/projects/08403431d26a466c955f6769705ba2c1/users/9d182958787749ff9b4bbad75adc4e91/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:02.453040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037702ab-13ec-44ad-83c2-ff621d1246ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=08403431d26a466c955f6769705ba2c1, user_id=9d182958787749ff9b4bbad75adc4e91, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:02.455766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037702ab-13ec-44ad-83c2-ff621d1246ce admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:02.529379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1270/2551] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:28:02 2022] PUT /identity/v3/projects/08403431d26a466c955f6769705ba2c1/users/9d182958787749ff9b4bbad75adc4e91/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:02.533233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c263e-7ea4-409f-b82d-8ceecab31a55 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:02.543738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c263e-7ea4-409f-b82d-8ceecab31a55 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:02.599730 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47824928-ceb0-430f-b8f9-c7e4d9f327ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:02.600280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47824928-ceb0-430f-b8f9-c7e4d9f327ee admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:02.600724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47824928-ceb0-430f-b8f9-c7e4d9f327ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:02.600995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47824928-ceb0-430f-b8f9-c7e4d9f327ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:02.604206 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47824928-ceb0-430f-b8f9-c7e4d9f327ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:02.606996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47824928-ceb0-430f-b8f9-c7e4d9f327ee admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:02.618130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1282/2552] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:02.622275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037702ab-13ec-44ad-83c2-ff621d1246ce admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:02.634107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037702ab-13ec-44ad-83c2-ff621d1246ce admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:02.638835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:02.639484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:02.639759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:02.640029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] PATH_INFO: `/v3/projects/08403431d26a466c955f6769705ba2c1/users/9d182958787749ff9b4bbad75adc4e91/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:02.641925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=08403431d26a466c955f6769705ba2c1, user_id=9d182958787749ff9b4bbad75adc4e91, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:02.644633 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:02.655658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1271/2553] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:28:02 2022] PUT /identity/v3/projects/08403431d26a466c955f6769705ba2c1/users/9d182958787749ff9b4bbad75adc4e91/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:02.670806 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13cc2-e1c8-4f06-add3-dfd2d6560c02 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:02.671086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13cc2-e1c8-4f06-add3-dfd2d6560c02 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:02.672030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13cc2-e1c8-4f06-add3-dfd2d6560c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:02.701633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-00b13cc2-e1c8-4f06-add3-dfd2d6560c02 None None] MFA Rules not processed for user `9d182958787749ff9b4bbad75adc4e91`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:02.702988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-00b13cc2-e1c8-4f06-add3-dfd2d6560c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:02.778750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1283/2554] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:02.789542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:02.796335 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:02.852933 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46542b7f-2f2d-4c83-8bf7-00c88cde2e7a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:02.856762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552f4a5-5cb1-4d91-988f-f754d5b45743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:02.857366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552f4a5-5cb1-4d91-988f-f754d5b45743 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:02.857643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552f4a5-5cb1-4d91-988f-f754d5b45743 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:02.857913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552f4a5-5cb1-4d91-988f-f754d5b45743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:02.915822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552f4a5-5cb1-4d91-988f-f754d5b45743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:02.917286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552f4a5-5cb1-4d91-988f-f754d5b45743 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:02.926200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1272/2555] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:08.014137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b13cc2-e1c8-4f06-add3-dfd2d6560c02 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:08.015582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-00b13cc2-e1c8-4f06-add3-dfd2d6560c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:08.028280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b13cc2-e1c8-4f06-add3-dfd2d6560c02 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:08.031988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed2050-1883-4c4c-bea1-1bdf867e1bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:08.032616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed2050-1883-4c4c-bea1-1bdf867e1bdb service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:08.032889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed2050-1883-4c4c-bea1-1bdf867e1bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:08.033155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed2050-1883-4c4c-bea1-1bdf867e1bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:08.042840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed2050-1883-4c4c-bea1-1bdf867e1bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:08.044259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed2050-1883-4c4c-bea1-1bdf867e1bdb service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:08.054316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1284/2556] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:12.922101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:13.823200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552f4a5-5cb1-4d91-988f-f754d5b45743 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:13.892084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552f4a5-5cb1-4d91-988f-f754d5b45743 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:13.896625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:13.897822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:13.898095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:13.898390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] PATH_INFO: `/v3/users/0836c198b01248bdaa3d0a1854237210` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:13.899542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0836c198b01248bdaa3d0a1854237210)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:13.901483 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:14.110897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0836c198b01248bdaa3d0a1854237210'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:14.152220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0836c198b01248bdaa3d0a1854237210'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:14.152930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0836c198b01248bdaa3d0a1854237210'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:14.164625 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0836c198b01248bdaa3d0a1854237210'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:14.173229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0836c198b01248bdaa3d0a1854237210'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:14.179922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1273/2557] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:28:13 2022] DELETE /identity/v3/users/0836c198b01248bdaa3d0a1854237210 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:14.188072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5e6c8-b887-4f24-9627-57ca81eed9b0 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:14.188547 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5e6c8-b887-4f24-9627-57ca81eed9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:14.189134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5e6c8-b887-4f24-9627-57ca81eed9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:14.211301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-8ed5e6c8-b887-4f24-9627-57ca81eed9b0 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:14.212551 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8ed5e6c8-b887-4f24-9627-57ca81eed9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:14.251853 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:14.274632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1285/2558] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:14.284380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:14.285851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:14.354430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23adfa-64f3-44e2-bb8d-4264e2beead6 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:14.358068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0358bfd-448f-4905-acb9-c425c6a824e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:14.358707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0358bfd-448f-4905-acb9-c425c6a824e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:14.358985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0358bfd-448f-4905-acb9-c425c6a824e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:14.359250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0358bfd-448f-4905-acb9-c425c6a824e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:14.369419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0358bfd-448f-4905-acb9-c425c6a824e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:14.370988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0358bfd-448f-4905-acb9-c425c6a824e4 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:14.381533 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1274/2559] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:28:14.799976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5e6c8-b887-4f24-9627-57ca81eed9b0 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:14.801613 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8ed5e6c8-b887-4f24-9627-57ca81eed9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:14.823467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5e6c8-b887-4f24-9627-57ca81eed9b0 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:14.830757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:14.831364 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:14.831726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:14.832003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] PATH_INFO: `/v3/projects/e4d69be9c90c441cbab25bc41031eeae` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:14.833163 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d69be9c90c441cbab25bc41031eeae)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:14.836664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:14.932954 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d69be9c90c441cbab25bc41031eeae', 'request_id': 'req-62ce1d44-5670-4783-ae64-003445b915fc'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.002111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d69be9c90c441cbab25bc41031eeae', 'request_id': 'req-62ce1d44-5670-4783-ae64-003445b915fc'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.010497 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d69be9c90c441cbab25bc41031eeae', 'request_id': 'req-62ce1d44-5670-4783-ae64-003445b915fc'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.020919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] The token cache is being invalidate because project e4d69be9c90c441cbab25bc41031eeae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:28:15.021443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.022254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.027036 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1286/2560] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:28:14 2022] DELETE /identity/v3/projects/e4d69be9c90c441cbab25bc41031eeae => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:15.031247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0358bfd-448f-4905-acb9-c425c6a824e4 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:15.041172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0358bfd-448f-4905-acb9-c425c6a824e4 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:15.060741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:15.098912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:15.099542 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:15.099838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:15.100119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] PATH_INFO: `/v3/users/9d182958787749ff9b4bbad75adc4e91` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:15.101326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d182958787749ff9b4bbad75adc4e91)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:15.103279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:15.285900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d182958787749ff9b4bbad75adc4e91'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.345583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d182958787749ff9b4bbad75adc4e91'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.346198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d182958787749ff9b4bbad75adc4e91'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.356571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d182958787749ff9b4bbad75adc4e91'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.370652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e50ee4e0-62f7-44dc-a7d4-fae2c51182a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d182958787749ff9b4bbad75adc4e91'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:15.377215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1275/2561] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:28:15 2022] DELETE /identity/v3/users/9d182958787749ff9b4bbad75adc4e91 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:28:15.908672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:15.910358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:15.924798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce1d44-5670-4783-ae64-003445b915fc admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:16.003056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:16.003056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.003056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.003056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] PATH_INFO: `/v3/projects/08403431d26a466c955f6769705ba2c1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.003565 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08403431d26a466c955f6769705ba2c1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:16.004510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:16.139435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08403431d26a466c955f6769705ba2c1', 'request_id': 'req-4411b62c-5b08-411d-8411-9788fe036f19'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:16.220988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08403431d26a466c955f6769705ba2c1', 'request_id': 'req-4411b62c-5b08-411d-8411-9788fe036f19'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:16.229503 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08403431d26a466c955f6769705ba2c1', 'request_id': 'req-4411b62c-5b08-411d-8411-9788fe036f19'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:16.233081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:16.244766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] The token cache is being invalidate because project 08403431d26a466c955f6769705ba2c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:28:16.245062 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:16.245870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4411b62c-5b08-411d-8411-9788fe036f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:16.248124 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1287/2562] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:28:15 2022] DELETE /identity/v3/projects/08403431d26a466c955f6769705ba2c1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:16.254070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39ba94-5dce-47c4-984f-8393416ad874 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.256799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39ba94-5dce-47c4-984f-8393416ad874 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.256799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39ba94-5dce-47c4-984f-8393416ad874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.278174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4c39ba94-5dce-47c4-984f-8393416ad874 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:16.279603 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4c39ba94-5dce-47c4-984f-8393416ad874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:16.359539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1276/2563] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:28:16.369321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4879ae-9031-40d5-bcc5-d396b0b25a68 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.369605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4879ae-9031-40d5-bcc5-d396b0b25a68 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.369882 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4879ae-9031-40d5-bcc5-d396b0b25a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.389738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3e4879ae-9031-40d5-bcc5-d396b0b25a68 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:16.390983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3e4879ae-9031-40d5-bcc5-d396b0b25a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:16.399716 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1288/2564] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:16.403917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39ba94-5dce-47c4-984f-8393416ad874 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:16.413736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39ba94-5dce-47c4-984f-8393416ad874 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:16.420912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:16.421627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.422210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.422626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.423296 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:28:16.423596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:16.425769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:16.445600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1277/2565] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:28:16.451419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4879ae-9031-40d5-bcc5-d396b0b25a68 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:16.461473 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4879ae-9031-40d5-bcc5-d396b0b25a68 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:16.468333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38097db5-4ad3-476e-87ba-863dc42b71a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:16.468928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38097db5-4ad3-476e-87ba-863dc42b71a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.469203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38097db5-4ad3-476e-87ba-863dc42b71a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.469469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38097db5-4ad3-476e-87ba-863dc42b71a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.470100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38097db5-4ad3-476e-87ba-863dc42b71a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:16.472044 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38097db5-4ad3-476e-87ba-863dc42b71a1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:16.555311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1289/2566] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:28:16 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:16.560091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:16.571707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54979f7e-2b6a-4e60-8c3c-fc66c459fdce admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:16.602154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:16.633529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1776e6-786a-46b4-a326-a8d128f0b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:16.636830 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1776e6-786a-46b4-a326-a8d128f0b38e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.637216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1776e6-786a-46b4-a326-a8d128f0b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.637557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1776e6-786a-46b4-a326-a8d128f0b38e admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.638331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1776e6-786a-46b4-a326-a8d128f0b38e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:16.640374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1776e6-786a-46b4-a326-a8d128f0b38e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:16.733648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1278/2567] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:28:16 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:16.735322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38097db5-4ad3-476e-87ba-863dc42b71a1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:16.747615 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38097db5-4ad3-476e-87ba-863dc42b71a1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:16.760926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a74a88-e3d8-4ddd-8f17-2473c8f24d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:16.761410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a74a88-e3d8-4ddd-8f17-2473c8f24d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.761736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a74a88-e3d8-4ddd-8f17-2473c8f24d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.762297 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a74a88-e3d8-4ddd-8f17-2473c8f24d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.762878 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a74a88-e3d8-4ddd-8f17-2473c8f24d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:16.764469 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a74a88-e3d8-4ddd-8f17-2473c8f24d70 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:16.779501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1290/2568] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:16.781300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1776e6-786a-46b4-a326-a8d128f0b38e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:16.802463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1776e6-786a-46b4-a326-a8d128f0b38e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:16.808101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa38d7f-c17e-49c7-86fd-a33e2d3970e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:16.809646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38d7f-c17e-49c7-86fd-a33e2d3970e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.809646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38d7f-c17e-49c7-86fd-a33e2d3970e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.809646 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38d7f-c17e-49c7-86fd-a33e2d3970e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.809932 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa38d7f-c17e-49c7-86fd-a33e2d3970e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:16.811544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa38d7f-c17e-49c7-86fd-a33e2d3970e3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:16.826179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1279/2569] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:16.832928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a74a88-e3d8-4ddd-8f17-2473c8f24d70 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:16.843479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a74a88-e3d8-4ddd-8f17-2473c8f24d70 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:16.848488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:16.849144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.849443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.849697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] PATH_INFO: `/v3/projects/80ff4b123c9d4f82aa97b661c22fc9ba/users/0c4451de16cd4764b1b8d5b1843a83fe/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.868104 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ff4b123c9d4f82aa97b661c22fc9ba, user_id=0c4451de16cd4764b1b8d5b1843a83fe, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:16.870951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:16.930372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1291/2570] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:28:16 2022] PUT /identity/v3/projects/80ff4b123c9d4f82aa97b661c22fc9ba/users/0c4451de16cd4764b1b8d5b1843a83fe/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:16.941723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57495da-2c36-4e8d-aa7d-867c957ee37a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:16.941945 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57495da-2c36-4e8d-aa7d-867c957ee37a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:16.942219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57495da-2c36-4e8d-aa7d-867c957ee37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:16.972558 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-b57495da-2c36-4e8d-aa7d-867c957ee37a None None] MFA Rules not processed for user `0c4451de16cd4764b1b8d5b1843a83fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:16.973896 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b57495da-2c36-4e8d-aa7d-867c957ee37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:17.056202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1280/2571] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:17.067928 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:17.070860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:17.140162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb874c-4aa8-49cc-acf4-154fd727c864 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:17.144013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1842c116caa4a5d9d75c8605eeec0aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:17.144620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:17.145015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:17.145273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:17.155530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:17.157034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:17.170499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1292/2572] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:20.223054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f910a-db92-45b2-9fd8-207551b9e527 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:20.224124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f910a-db92-45b2-9fd8-207551b9e527 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:20.224395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f910a-db92-45b2-9fd8-207551b9e527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:20.245286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bf4f910a-db92-45b2-9fd8-207551b9e527 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:20.246528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bf4f910a-db92-45b2-9fd8-207551b9e527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:20.309499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1281/2573] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:28:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:20.364114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:20.365833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:20.462680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f574b3-f59d-4254-b8a0-14dfe8a037b2 service glance] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:20.468372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6023830-f154-43e9-8aef-68a405b53f54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:20.468980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6023830-f154-43e9-8aef-68a405b53f54 service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:20.469260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6023830-f154-43e9-8aef-68a405b53f54 service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:20.469664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6023830-f154-43e9-8aef-68a405b53f54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:20.480789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6023830-f154-43e9-8aef-68a405b53f54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:20.482488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6023830-f154-43e9-8aef-68a405b53f54 service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:20.491907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1293/2574] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:28:27.297608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f910a-db92-45b2-9fd8-207551b9e527 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:27.297608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bf4f910a-db92-45b2-9fd8-207551b9e527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:27.326835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:27.337070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f910a-db92-45b2-9fd8-207551b9e527 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:27.340803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:27.341340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:27.341614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:27.341881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:27.343618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:27.420018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:27.421470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:27.432106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1282/2575] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:30.292126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bbef7-338a-43c1-bff5-8f06f5b9d16c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:30.292755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bbef7-338a-43c1-bff5-8f06f5b9d16c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:30.292755 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bbef7-338a-43c1-bff5-8f06f5b9d16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:30.319129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-157bbef7-338a-43c1-bff5-8f06f5b9d16c None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:30.320961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-157bbef7-338a-43c1-bff5-8f06f5b9d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:30.338527 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1294/2576] 10.222.0.20 () {58 vars in 967 bytes} [Fri Sep 9 23:28:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:30.359516 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:30.372127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c596e-689d-4e4b-9f27-4c1151022af1 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:30.379608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:30.380615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:30.381088 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:30.381508 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:30.415756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:30.417366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:30.436620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1283/2577] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:28:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:28:31.416796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bbef7-338a-43c1-bff5-8f06f5b9d16c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:31.419602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-157bbef7-338a-43c1-bff5-8f06f5b9d16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:31.441242 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:31.492903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bbef7-338a-43c1-bff5-8f06f5b9d16c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:31.498075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:31.498804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:31.498980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:31.499249 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:31.501057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:31.513074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:31.514673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:31.524980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1295/2578] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:31.574016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:31.574016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:31.585777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9bcbd-0d65-46f8-ad77-3076935bbea3 service swift] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:31.590334 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:31.591090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:31.591364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:31.591629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] PATH_INFO: `/v3/users/f097c4fc270748cc8d41d7bcaba672a7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:31.592734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f097c4fc270748cc8d41d7bcaba672a7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:31.594670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:31.799221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f097c4fc270748cc8d41d7bcaba672a7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:31.872367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f097c4fc270748cc8d41d7bcaba672a7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:31.873019 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f097c4fc270748cc8d41d7bcaba672a7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:31.886696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f097c4fc270748cc8d41d7bcaba672a7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:31.917501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-727f085c-34df-4a80-be4c-ad351d805a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f097c4fc270748cc8d41d7bcaba672a7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:31.925118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1284/2579] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:28:31 2022] DELETE /identity/v3/users/f097c4fc270748cc8d41d7bcaba672a7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:32.206678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:32.223897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8663acf-6441-4d56-a2c6-3bb792f764a7 service placement] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:32.312494 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:32.313075 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:32.313347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:32.313612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] PATH_INFO: `/v3/projects/5c3517b03576446a902816ab0bbff2e7` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:32.315078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c3517b03576446a902816ab0bbff2e7)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:32.317451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:32.429316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c3517b03576446a902816ab0bbff2e7', 'request_id': 'req-6a37d806-0656-4c53-9532-652f8e56de24'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:32.522813 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c3517b03576446a902816ab0bbff2e7', 'request_id': 'req-6a37d806-0656-4c53-9532-652f8e56de24'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:32.529976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c3517b03576446a902816ab0bbff2e7', 'request_id': 'req-6a37d806-0656-4c53-9532-652f8e56de24'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:32.544285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] The token cache is being invalidate because project 5c3517b03576446a902816ab0bbff2e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:28:32.544598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:32.545306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6a37d806-0656-4c53-9532-652f8e56de24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:32.550656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1296/2580] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:28:32 2022] DELETE /identity/v3/projects/5c3517b03576446a902816ab0bbff2e7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:32.554103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fd0eb-8772-465d-84f1-816600de6ca7 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:32.554698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fd0eb-8772-465d-84f1-816600de6ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:32.555434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7fd0eb-8772-465d-84f1-816600de6ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:32.582979 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-fb7fd0eb-8772-465d-84f1-816600de6ca7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:32.585888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fb7fd0eb-8772-465d-84f1-816600de6ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:32.640219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:32.658815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1285/2581] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:32.667119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30768ebf-2b94-44c4-9dae-1830d00cf89b None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:32.667405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30768ebf-2b94-44c4-9dae-1830d00cf89b None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:32.667674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30768ebf-2b94-44c4-9dae-1830d00cf89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:32.688653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-30768ebf-2b94-44c4-9dae-1830d00cf89b None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:32.689801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-30768ebf-2b94-44c4-9dae-1830d00cf89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:32.700763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1297/2582] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:32.705636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fd0eb-8772-465d-84f1-816600de6ca7 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:32.715885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7fd0eb-8772-465d-84f1-816600de6ca7 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:32.722252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:32.722696 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:32.722976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:32.723244 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:32.723890 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:28:32.724191 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:32.726588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:32.739789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1286/2583] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:28:32.744792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30768ebf-2b94-44c4-9dae-1830d00cf89b None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:32.754032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30768ebf-2b94-44c4-9dae-1830d00cf89b None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:32.762629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577348fc-a885-493c-801e-ad55b87187f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:32.763241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577348fc-a885-493c-801e-ad55b87187f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:32.763517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577348fc-a885-493c-801e-ad55b87187f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:32.763788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577348fc-a885-493c-801e-ad55b87187f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:32.764423 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577348fc-a885-493c-801e-ad55b87187f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:32.766379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577348fc-a885-493c-801e-ad55b87187f9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:32.854453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1298/2584] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:28:32 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:32.856458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:32.868154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5c740-90fe-4cf1-b0fe-eb998037e724 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:32.934862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a063a-5a01-46b1-9c70-f3ccd41d68ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:32.935936 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a063a-5a01-46b1-9c70-f3ccd41d68ab admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:32.936601 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a063a-5a01-46b1-9c70-f3ccd41d68ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:32.937021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a063a-5a01-46b1-9c70-f3ccd41d68ab admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:32.937821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a063a-5a01-46b1-9c70-f3ccd41d68ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:32.940290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a063a-5a01-46b1-9c70-f3ccd41d68ab admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:33.032729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1287/2585] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:28:32 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:28:33.037156 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577348fc-a885-493c-801e-ad55b87187f9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:33.048546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577348fc-a885-493c-801e-ad55b87187f9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:33.054160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f4504-6b35-4e1f-b79e-98cbc9461e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:33.054786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4504-6b35-4e1f-b79e-98cbc9461e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:33.055060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4504-6b35-4e1f-b79e-98cbc9461e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:33.055324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f4504-6b35-4e1f-b79e-98cbc9461e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:33.055947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f4504-6b35-4e1f-b79e-98cbc9461e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:33.057534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f4504-6b35-4e1f-b79e-98cbc9461e80 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:33.059880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:33.068057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1299/2586] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:33.072723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a063a-5a01-46b1-9c70-f3ccd41d68ab admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:33.088398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a063a-5a01-46b1-9c70-f3ccd41d68ab admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:33.088398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffa129-8d7f-4b6f-8aeb-142d8ecf166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:33.088398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffa129-8d7f-4b6f-8aeb-142d8ecf166f admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:33.088794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffa129-8d7f-4b6f-8aeb-142d8ecf166f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:33.088794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffa129-8d7f-4b6f-8aeb-142d8ecf166f admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:33.089301 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffa129-8d7f-4b6f-8aeb-142d8ecf166f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:33.090923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffa129-8d7f-4b6f-8aeb-142d8ecf166f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:33.102692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1288/2587] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:33.106960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f4504-6b35-4e1f-b79e-98cbc9461e80 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:33.118983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f4504-6b35-4e1f-b79e-98cbc9461e80 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:33.133298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:33.133988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:33.134271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:33.134568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] PATH_INFO: `/v3/projects/9d57c38134f1484d9b208de1b9d8e26e/users/592da799f0c540d68a2cc6d54006467c/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:33.155601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d57c38134f1484d9b208de1b9d8e26e, user_id=592da799f0c540d68a2cc6d54006467c, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:33.158463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:33.250037 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1300/2588] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:28:33 2022] PUT /identity/v3/projects/9d57c38134f1484d9b208de1b9d8e26e/users/592da799f0c540d68a2cc6d54006467c/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:28:33.256498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a450d-a2ca-41ac-b5ee-4ce18be95aba None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:33.257056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a450d-a2ca-41ac-b5ee-4ce18be95aba None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:33.257374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a450d-a2ca-41ac-b5ee-4ce18be95aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:33.279249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-917a450d-a2ca-41ac-b5ee-4ce18be95aba None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:33.280459 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-917a450d-a2ca-41ac-b5ee-4ce18be95aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:33.364308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1289/2589] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:33.372107 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:33.377303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:33.453904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8dc41-0564-4b6b-8232-a8a7ccefb1a1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:33.457734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:33.458608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:33.459086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:33.459522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:33.469336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:33.470851 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:33.481993 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1301/2590] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:40.030553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5a0b1-df6d-4562-85f1-b847cf3a2d3e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:40.031993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5a0b1-df6d-4562-85f1-b847cf3a2d3e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:40.032690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5a0b1-df6d-4562-85f1-b847cf3a2d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:40.119856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-17b5a0b1-df6d-4562-85f1-b847cf3a2d3e None None] MFA Rules not processed for user `592da799f0c540d68a2cc6d54006467c`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:40.119856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-17b5a0b1-df6d-4562-85f1-b847cf3a2d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:40.202392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:40.242775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1290/2591] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:40.253085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:40.260680 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:40.348777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25681a-0937-4263-820b-b17618a5c2f5 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:40.355737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:40.358064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:40.358064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:40.358064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:40.366231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:40.367860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:40.377854 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1302/2592] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:42.152092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5a0b1-df6d-4562-85f1-b847cf3a2d3e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:42.154467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-17b5a0b1-df6d-4562-85f1-b847cf3a2d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:42.257135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5a0b1-df6d-4562-85f1-b847cf3a2d3e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:42.262893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:42.263595 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:42.263940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:42.264277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:42.266281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:42.285268 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:42.288150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:42.289173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:42.291132 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:42.309836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1291/2593] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:42.312953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a470da7-e524-40d1-8d89-c9d6aa4af1fc service nova] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:42.321841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:42.323548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:42.323900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:42.324239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] PATH_INFO: `/v3/users/c4129e5f14794a619acdaf5c56a02cb8` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:42.325575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4129e5f14794a619acdaf5c56a02cb8)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:42.327591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:42.566320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4129e5f14794a619acdaf5c56a02cb8'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:42.568365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4129e5f14794a619acdaf5c56a02cb8'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:42.672521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4129e5f14794a619acdaf5c56a02cb8'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:42.675697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:42.689647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4129e5f14794a619acdaf5c56a02cb8'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:42.699281 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2c187edd-1038-4a7e-8d04-39b5c79b5812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4129e5f14794a619acdaf5c56a02cb8'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:42.705311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1303/2594] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:28:42 2022] DELETE /identity/v3/users/c4129e5f14794a619acdaf5c56a02cb8 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:43.089318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:43.108256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00072cb-8a6c-47be-bc3c-935d434569f2 service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:43.188339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:43.189974 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:43.190395 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:43.190793 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] PATH_INFO: `/v3/projects/28d0bb1d968841c2986b5eb164af953b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:43.192363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28d0bb1d968841c2986b5eb164af953b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:43.194935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:43.276988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15494-91f8-4760-b02f-e813b50649eb None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:43.277492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15494-91f8-4760-b02f-e813b50649eb None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:43.277774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d15494-91f8-4760-b02f-e813b50649eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:43.299437 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-24d15494-91f8-4760-b02f-e813b50649eb None None] MFA Rules not processed for user `592da799f0c540d68a2cc6d54006467c`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:43.300661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-24d15494-91f8-4760-b02f-e813b50649eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:43.369504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28d0bb1d968841c2986b5eb164af953b', 'request_id': 'req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.407791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1304/2595] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:28:43.426772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28d0bb1d968841c2986b5eb164af953b', 'request_id': 'req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.434518 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28d0bb1d968841c2986b5eb164af953b', 'request_id': 'req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.454986 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] The token cache is being invalidate because project 28d0bb1d968841c2986b5eb164af953b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:28:43.455360 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.456664 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.459723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1292/2596] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:28:43 2022] DELETE /identity/v3/projects/28d0bb1d968841c2986b5eb164af953b => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:43.463943 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d15494-91f8-4760-b02f-e813b50649eb None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:43.465531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-24d15494-91f8-4760-b02f-e813b50649eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:43.481359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d15494-91f8-4760-b02f-e813b50649eb None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:43.562865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:43.563507 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:43.563782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:43.564664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] PATH_INFO: `/v3/users/0e082fd6f18849029c18681675d05fb0` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:43.566073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e082fd6f18849029c18681675d05fb0)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:43.568734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:43.767656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e082fd6f18849029c18681675d05fb0'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.769428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e082fd6f18849029c18681675d05fb0'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.855192 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e082fd6f18849029c18681675d05fb0'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.871935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e082fd6f18849029c18681675d05fb0'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.894312 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9d7c449b-f110-44bb-8018-05a50c6274fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e082fd6f18849029c18681675d05fb0'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:43.896399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:43.903883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1305/2597] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:28:43 2022] DELETE /identity/v3/users/0e082fd6f18849029c18681675d05fb0 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:44.320630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:44.324199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:44.401161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80936d-8d62-4757-9b1e-e0fcaf2f8d24 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:44.410765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:44.561881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:44.562810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:44.563488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:44.563904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] PATH_INFO: `/v3/projects/74b5ccd665c245debc0f7bfe5a97237e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:44.565227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b5ccd665c245debc0f7bfe5a97237e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:44.573555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:44.776340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b5ccd665c245debc0f7bfe5a97237e', 'request_id': 'req-e53a25a4-331f-47cb-92a1-0870b1266895'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:44.876959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b5ccd665c245debc0f7bfe5a97237e', 'request_id': 'req-e53a25a4-331f-47cb-92a1-0870b1266895'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:44.893817 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b5ccd665c245debc0f7bfe5a97237e', 'request_id': 'req-e53a25a4-331f-47cb-92a1-0870b1266895'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:44.925295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] The token cache is being invalidate because project 74b5ccd665c245debc0f7bfe5a97237e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:28:44.925590 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:44.926267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-e53a25a4-331f-47cb-92a1-0870b1266895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:44.928240 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1293/2598] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:28:44 2022] DELETE /identity/v3/projects/74b5ccd665c245debc0f7bfe5a97237e => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:44.933489 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b0fb2-d373-4cc0-96a8-7e303fa8381c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:44.934014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b0fb2-d373-4cc0-96a8-7e303fa8381c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:44.934014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b0fb2-d373-4cc0-96a8-7e303fa8381c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:44.964310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e64b0fb2-d373-4cc0-96a8-7e303fa8381c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:44.965557 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e64b0fb2-d373-4cc0-96a8-7e303fa8381c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:45.074639 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1306/2599] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:45.083808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac12b5-f7c9-462e-a28d-6b88d0cce42a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.084319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac12b5-f7c9-462e-a28d-6b88d0cce42a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.084600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac12b5-f7c9-462e-a28d-6b88d0cce42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.122683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0dac12b5-f7c9-462e-a28d-6b88d0cce42a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:45.124433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0dac12b5-f7c9-462e-a28d-6b88d0cce42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:45.139993 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1294/2600] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:45.140520 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b0fb2-d373-4cc0-96a8-7e303fa8381c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:45.151510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b0fb2-d373-4cc0-96a8-7e303fa8381c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:45.162916 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:45.163789 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.164068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.164336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.165042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:28:45.165345 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:45.167019 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:45.186568 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1307/2601] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 09 23:28:45.191839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac12b5-f7c9-462e-a28d-6b88d0cce42a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:45.210291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac12b5-f7c9-462e-a28d-6b88d0cce42a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:45.215321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff818d-2b71-48e8-a831-60d33ae948bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:45.216252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff818d-2b71-48e8-a831-60d33ae948bf admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.216252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff818d-2b71-48e8-a831-60d33ae948bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.216460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff818d-2b71-48e8-a831-60d33ae948bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.217075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff818d-2b71-48e8-a831-60d33ae948bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:45.219228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff818d-2b71-48e8-a831-60d33ae948bf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:45.330686 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1295/2602] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:28:45 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:45.335035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:45.348392 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ada3c7-6e3b-4d5f-a926-52863522a469 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:45.428449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3009e96-6e20-4c0b-a508-d301b82ac1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:45.429093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3009e96-6e20-4c0b-a508-d301b82ac1c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.429366 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3009e96-6e20-4c0b-a508-d301b82ac1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.429660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3009e96-6e20-4c0b-a508-d301b82ac1c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.430754 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3009e96-6e20-4c0b-a508-d301b82ac1c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:45.432926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3009e96-6e20-4c0b-a508-d301b82ac1c3 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:45.528260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1308/2603] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:28:45 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:45.532227 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff818d-2b71-48e8-a831-60d33ae948bf admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:45.544537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4d002-8789-4483-8fb6-9fd455597c1f None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.544828 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4d002-8789-4483-8fb6-9fd455597c1f None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.545171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4d002-8789-4483-8fb6-9fd455597c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.552285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff818d-2b71-48e8-a831-60d33ae948bf admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:45.562553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3b3f4-0421-48f2-afbb-d8d060843efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:45.562553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3b3f4-0421-48f2-afbb-d8d060843efa admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.562553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3b3f4-0421-48f2-afbb-d8d060843efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.562553 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3b3f4-0421-48f2-afbb-d8d060843efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.563438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3b3f4-0421-48f2-afbb-d8d060843efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:45.564300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3b3f4-0421-48f2-afbb-d8d060843efa admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:45.577893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1296/2604] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:45.582477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3b3f4-0421-48f2-afbb-d8d060843efa admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:45.582923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-51f4d002-8789-4483-8fb6-9fd455597c1f None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:45.584947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-51f4d002-8789-4483-8fb6-9fd455597c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:45.600011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3b3f4-0421-48f2-afbb-d8d060843efa admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:45.608810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea732dff-43dd-4eb2-a5cc-c9ced2c94cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:45.609954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea732dff-43dd-4eb2-a5cc-c9ced2c94cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.610560 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea732dff-43dd-4eb2-a5cc-c9ced2c94cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.610875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea732dff-43dd-4eb2-a5cc-c9ced2c94cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.612020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea732dff-43dd-4eb2-a5cc-c9ced2c94cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:45.614151 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea732dff-43dd-4eb2-a5cc-c9ced2c94cb1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:45.630959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1297/2605] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:28:45.632059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea732dff-43dd-4eb2-a5cc-c9ced2c94cb1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:45.644997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea732dff-43dd-4eb2-a5cc-c9ced2c94cb1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:45.651715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fbc7b-51a6-45bb-bff2-a96ee3521beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:45.652569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fbc7b-51a6-45bb-bff2-a96ee3521beb admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.652984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fbc7b-51a6-45bb-bff2-a96ee3521beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.653433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fbc7b-51a6-45bb-bff2-a96ee3521beb admin admin] PATH_INFO: `/v3/projects/8042eb99cd7047d4912b2f4b1e50936e/users/8c81c0af6f084b8db165fd51667b3b54/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.662572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1309/2606] 10.222.0.20 () {58 vars in 1009 bytes} [Fri Sep 9 23:28:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:28:45.676224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4d002-8789-4483-8fb6-9fd455597c1f None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:45.677962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-51f4d002-8789-4483-8fb6-9fd455597c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:45.678829 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fbc7b-51a6-45bb-bff2-a96ee3521beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8042eb99cd7047d4912b2f4b1e50936e, user_id=8c81c0af6f084b8db165fd51667b3b54, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:45.683212 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fbc7b-51a6-45bb-bff2-a96ee3521beb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:45.702941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:45.755445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4d002-8789-4483-8fb6-9fd455597c1f None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:45.767322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:45.767322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.767322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.767322 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.776458 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1298/2607] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:28:45 2022] PUT /identity/v3/projects/8042eb99cd7047d4912b2f4b1e50936e/users/8c81c0af6f084b8db165fd51667b3b54/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:45.777106 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:45.778913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:45.782221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ef9f8-807d-4917-92ff-d79b1680c323 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:45.782680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ef9f8-807d-4917-92ff-d79b1680c323 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:45.782929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ef9f8-807d-4917-92ff-d79b1680c323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:45.788947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1310/2608] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:28:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 09 23:28:45.819969 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3d2ef9f8-807d-4917-92ff-d79b1680c323 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:45.826075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3d2ef9f8-807d-4917-92ff-d79b1680c323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:45.840367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:45.898763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1299/2609] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:28:45.909814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:45.910914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:46.025687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fef1a-af8b-47df-8ff5-6dc97ad2fb5a service swift] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:46.104051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa8073-3257-4357-ab6d-3f1bab84447c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:46.105377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa8073-3257-4357-ab6d-3f1bab84447c service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:46.106073 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa8073-3257-4357-ab6d-3f1bab84447c service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:46.106748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa8073-3257-4357-ab6d-3f1bab84447c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:46.121667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa8073-3257-4357-ab6d-3f1bab84447c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:46.123485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa8073-3257-4357-ab6d-3f1bab84447c service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:46.136994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1311/2610] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:28:53.726970 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2248d1-b393-4b50-b219-1aa51592f4e6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:53.727884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2248d1-b393-4b50-b219-1aa51592f4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:53.728341 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2248d1-b393-4b50-b219-1aa51592f4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:53.771692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ac2248d1-b393-4b50-b219-1aa51592f4e6 None None] MFA Rules not processed for user `8c81c0af6f084b8db165fd51667b3b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:53.773273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ac2248d1-b393-4b50-b219-1aa51592f4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:53.946362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1300/2611] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:53.951291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa8073-3257-4357-ab6d-3f1bab84447c service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:53.977015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa8073-3257-4357-ab6d-3f1bab84447c service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:53.995823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b684c-e7fe-450b-af38-140bbfbf516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:53.998849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b684c-e7fe-450b-af38-140bbfbf516c admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:54.001014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b684c-e7fe-450b-af38-140bbfbf516c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:54.001317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b684c-e7fe-450b-af38-140bbfbf516c admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:54.003762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b684c-e7fe-450b-af38-140bbfbf516c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:54.006257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b684c-e7fe-450b-af38-140bbfbf516c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:54.231277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1312/2612] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:28:53 2022] POST /identity/v3/projects => generated 379 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:54.235512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2248d1-b393-4b50-b219-1aa51592f4e6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:54.252680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2248d1-b393-4b50-b219-1aa51592f4e6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:54.371814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11192d-a442-473f-b534-a8113b4dcb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:54.372488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11192d-a442-473f-b534-a8113b4dcb00 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:54.372772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11192d-a442-473f-b534-a8113b4dcb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:54.373111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11192d-a442-473f-b534-a8113b4dcb00 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:54.373765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11192d-a442-473f-b534-a8113b4dcb00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:54.375814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11192d-a442-473f-b534-a8113b4dcb00 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:54.509189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1301/2613] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:28:54 2022] POST /identity/v3/users => generated 296 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:28:54.575627 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b684c-e7fe-450b-af38-140bbfbf516c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:54.587818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b684c-e7fe-450b-af38-140bbfbf516c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:54.592599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22104f1c-754f-481a-89a4-fc9acc608dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:54.593221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22104f1c-754f-481a-89a4-fc9acc608dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:54.593523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22104f1c-754f-481a-89a4-fc9acc608dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:54.593793 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22104f1c-754f-481a-89a4-fc9acc608dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:54.594475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22104f1c-754f-481a-89a4-fc9acc608dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:54.596143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22104f1c-754f-481a-89a4-fc9acc608dc1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:54.619831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1313/2614] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:54.624776 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11192d-a442-473f-b534-a8113b4dcb00 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:54.637585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11192d-a442-473f-b534-a8113b4dcb00 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:54.642870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af64d63-283c-4b2a-aa2d-7d4d1013e0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:54.643578 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af64d63-283c-4b2a-aa2d-7d4d1013e0bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:54.643845 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af64d63-283c-4b2a-aa2d-7d4d1013e0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:54.644129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af64d63-283c-4b2a-aa2d-7d4d1013e0bf admin admin] PATH_INFO: `/v3/projects/a5245d16c18f43ac9a3c08be6e0fcfd8/users/6164c0b114624a4b99ea533894a72f0b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:54.662029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af64d63-283c-4b2a-aa2d-7d4d1013e0bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5245d16c18f43ac9a3c08be6e0fcfd8, user_id=6164c0b114624a4b99ea533894a72f0b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:54.664789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af64d63-283c-4b2a-aa2d-7d4d1013e0bf admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:54.769136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1302/2615] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:28:54 2022] PUT /identity/v3/projects/a5245d16c18f43ac9a3c08be6e0fcfd8/users/6164c0b114624a4b99ea533894a72f0b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:28:54.776781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22104f1c-754f-481a-89a4-fc9acc608dc1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:54.793130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22104f1c-754f-481a-89a4-fc9acc608dc1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:54.846830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:54.894622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:54.894622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:54.894622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:54.894622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:54.895625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:54.897460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:54.919212 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1314/2616] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:28:54.923889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af64d63-283c-4b2a-aa2d-7d4d1013e0bf admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:54.938976 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af64d63-283c-4b2a-aa2d-7d4d1013e0bf admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:54.962488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425613c-5aa3-4b54-b558-cae2f33cf56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:54.963447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425613c-5aa3-4b54-b558-cae2f33cf56d admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:54.963943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425613c-5aa3-4b54-b558-cae2f33cf56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:54.964370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425613c-5aa3-4b54-b558-cae2f33cf56d admin admin] PATH_INFO: `/v3/projects/a5245d16c18f43ac9a3c08be6e0fcfd8/users/6164c0b114624a4b99ea533894a72f0b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:54.983174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425613c-5aa3-4b54-b558-cae2f33cf56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5245d16c18f43ac9a3c08be6e0fcfd8, user_id=6164c0b114624a4b99ea533894a72f0b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:54.987256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425613c-5aa3-4b54-b558-cae2f33cf56d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:55.028913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1303/2617] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:28:54 2022] PUT /identity/v3/projects/a5245d16c18f43ac9a3c08be6e0fcfd8/users/6164c0b114624a4b99ea533894a72f0b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:56.725125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:56.725125 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:56.744169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22fc80-9f80-4a3b-b8d2-f1d0edc5cf0b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:56.897049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:56.897049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:56.897465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:56.897465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] PATH_INFO: `/v3/users/0c4451de16cd4764b1b8d5b1843a83fe` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:56.901178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4451de16cd4764b1b8d5b1843a83fe)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:56.903179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:57.170640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4451de16cd4764b1b8d5b1843a83fe'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:57.179303 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4451de16cd4764b1b8d5b1843a83fe'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:57.306812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4451de16cd4764b1b8d5b1843a83fe'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:57.340863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4451de16cd4764b1b8d5b1843a83fe'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:57.351410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4451de16cd4764b1b8d5b1843a83fe'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:57.358386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1315/2618] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:28:56 2022] DELETE /identity/v3/users/0c4451de16cd4764b1b8d5b1843a83fe => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:28:57.362838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a05438-35c1-40e9-93a8-8eca74192d1e None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:57.363397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a05438-35c1-40e9-93a8-8eca74192d1e None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:57.363397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a05438-35c1-40e9-93a8-8eca74192d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:57.372068 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:57.384379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-08a05438-35c1-40e9-93a8-8eca74192d1e None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:28:57.385833 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-08a05438-35c1-40e9-93a8-8eca74192d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:57.526498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1304/2619] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:28:57.540689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:57.543929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:57.694961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3af80-d06a-42b5-8f04-1b26384245b3 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:57.699614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cdb18-7531-4443-83d4-4c85f5cffe08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:57.700458 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cdb18-7531-4443-83d4-4c85f5cffe08 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:57.700911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cdb18-7531-4443-83d4-4c85f5cffe08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:57.701347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cdb18-7531-4443-83d4-4c85f5cffe08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:57.709279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:28:57.716264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cdb18-7531-4443-83d4-4c85f5cffe08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:57.719030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cdb18-7531-4443-83d4-4c85f5cffe08 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:57.733909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1316/2620] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:28:58.609036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a05438-35c1-40e9-93a8-8eca74192d1e None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:28:58.609036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-08a05438-35c1-40e9-93a8-8eca74192d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:28:58.639314 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a05438-35c1-40e9-93a8-8eca74192d1e None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:28:58.644055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:28:58.645007 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:28:58.645393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:28:58.645975 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] PATH_INFO: `/v3/projects/80ff4b123c9d4f82aa97b661c22fc9ba` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:28:58.648203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ff4b123c9d4f82aa97b661c22fc9ba)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:28:58.650634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:28:58.841455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ff4b123c9d4f82aa97b661c22fc9ba', 'request_id': 'req-ca7a230e-717f-48c9-9624-e729986a1536'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:58.900596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ff4b123c9d4f82aa97b661c22fc9ba', 'request_id': 'req-ca7a230e-717f-48c9-9624-e729986a1536'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:58.913847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ff4b123c9d4f82aa97b661c22fc9ba', 'request_id': 'req-ca7a230e-717f-48c9-9624-e729986a1536'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:58.929480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] The token cache is being invalidate because project 80ff4b123c9d4f82aa97b661c22fc9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:28:58.929941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:58.930736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:28:58.932493 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1305/2621] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:28:58 2022] DELETE /identity/v3/projects/80ff4b123c9d4f82aa97b661c22fc9ba => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:29:02.656428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e271b-062c-4d8c-b0e5-24764509ed6d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:02.656909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e271b-062c-4d8c-b0e5-24764509ed6d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:02.657109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e271b-062c-4d8c-b0e5-24764509ed6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:02.690335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c90e271b-062c-4d8c-b0e5-24764509ed6d None None] MFA Rules not processed for user `6164c0b114624a4b99ea533894a72f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:29:02.691607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c90e271b-062c-4d8c-b0e5-24764509ed6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:02.771245 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1317/2622] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:29:02 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:29:02.785621 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:02.787500 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:02.882298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a230e-717f-48c9-9624-e729986a1536 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:02.890666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:02.891404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:02.891563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:02.891832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:02.893627 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:02.935405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:02.987264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:02.987594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:03.000800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1306/2623] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:29:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:29:03.340670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e23a92-9ad1-44b4-bba9-a1815eba0b81 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:03.340960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e23a92-9ad1-44b4-bba9-a1815eba0b81 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:03.341239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e23a92-9ad1-44b4-bba9-a1815eba0b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:03.370712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-26e23a92-9ad1-44b4-bba9-a1815eba0b81 None None] MFA Rules not processed for user `2c8bfd34442b48a78600f568b534eaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:29:03.371925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-26e23a92-9ad1-44b4-bba9-a1815eba0b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:03.435738 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1318/2624] 10.222.0.20 () {58 vars in 967 bytes} [Fri Sep 9 23:29:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:29:03.452617 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:03.454221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:03.532702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798127-1746-4e9b-bdb7-665327262c22 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:03.538014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46642e5-8e07-4009-8619-d1c4b0499e88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '135696b09c2c4f809c49e0af200f32de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:03.539005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46642e5-8e07-4009-8619-d1c4b0499e88 service swift] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:03.539330 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46642e5-8e07-4009-8619-d1c4b0499e88 service swift] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:03.539600 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46642e5-8e07-4009-8619-d1c4b0499e88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:03.551418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46642e5-8e07-4009-8619-d1c4b0499e88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:03.555887 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46642e5-8e07-4009-8619-d1c4b0499e88 service swift] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:03.562194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1307/2625] 10.222.0.20 () {60 vars in 1384 bytes} [Fri Sep 9 23:29:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 09 23:29:13.428116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=62 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:13.558813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=48 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:14.971865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e23a92-9ad1-44b4-bba9-a1815eba0b81 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:14.973272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-26e23a92-9ad1-44b4-bba9-a1815eba0b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:15.135269 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e23a92-9ad1-44b4-bba9-a1815eba0b81 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:15.146668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:15.146668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:15.146668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:15.146668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:15.151940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:15.159243 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:15.160850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:15.171264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1319/2626] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:29:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:29:22.751971 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537f99-3705-4d76-a950-e915a38aa778 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:22.752411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537f99-3705-4d76-a950-e915a38aa778 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:22.752507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537f99-3705-4d76-a950-e915a38aa778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:22.771705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-00537f99-3705-4d76-a950-e915a38aa778 None None] MFA Rules not processed for user `8c81c0af6f084b8db165fd51667b3b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:29:22.772919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-00537f99-3705-4d76-a950-e915a38aa778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:22.780840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1308/2627] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:29:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:29:25.166675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=63 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:32.766104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:41.536660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:41.538100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:41.614103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42925dfd-df32-49d1-b410-c905e0007ca4 service placement] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:41.617714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:41.618298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:41.618596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:41.618861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:41.628501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:41.629898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:41.640199 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1320/2628] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:29:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:29:51.634930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=35 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:52.950803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537f99-3705-4d76-a950-e915a38aa778 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:52.960688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537f99-3705-4d76-a950-e915a38aa778 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:52.964924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:52.965596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:52.965868 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:52.966242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:52.967954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:53.053789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:53.055288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:53.078771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1309/2629] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:29:56.700949 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:56.702445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:56.717334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfac71-9469-4e7c-9f3b-cc23c0c1b379 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:56.721946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:56.722588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:56.722863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:56.723129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] PATH_INFO: `/v3/users/27be0979440d4ab081dae31790280eff` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:56.724351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27be0979440d4ab081dae31790280eff)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:56.726285 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:56.965795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27be0979440d4ab081dae31790280eff'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:56.966492 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27be0979440d4ab081dae31790280eff'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.051604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27be0979440d4ab081dae31790280eff'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.066567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27be0979440d4ab081dae31790280eff'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.074189 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b81a2444-0266-4f36-9838-4dd4c91649b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27be0979440d4ab081dae31790280eff'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.080195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1321/2630] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:29:56 2022] DELETE /identity/v3/users/27be0979440d4ab081dae31790280eff => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:29:57.333783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:57.345465 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e42da-857a-4c59-9f00-4e823e8e542a service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:57.438267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:57.439196 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:57.439604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:57.439850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] PATH_INFO: `/v3/projects/fbe6c3ad574e46fd9d3b37ead0542a18` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:57.441087 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbe6c3ad574e46fd9d3b37ead0542a18)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:57.443259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:57.583863 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbe6c3ad574e46fd9d3b37ead0542a18', 'request_id': 'req-49c14c57-b09c-407e-b56d-5d3fc65893a1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.648003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbe6c3ad574e46fd9d3b37ead0542a18', 'request_id': 'req-49c14c57-b09c-407e-b56d-5d3fc65893a1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.654159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbe6c3ad574e46fd9d3b37ead0542a18', 'request_id': 'req-49c14c57-b09c-407e-b56d-5d3fc65893a1'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.656126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:57.665501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] The token cache is being invalidate because project fbe6c3ad574e46fd9d3b37ead0542a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:29:57.666394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.667280 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-49c14c57-b09c-407e-b56d-5d3fc65893a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:29:57.668770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1310/2631] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:29:57 2022] DELETE /identity/v3/projects/fbe6c3ad574e46fd9d3b37ead0542a18 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:29:57.673622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53ff5-7262-49cb-bec8-15c1e87802af None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:57.675573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53ff5-7262-49cb-bec8-15c1e87802af None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:57.675573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f53ff5-7262-49cb-bec8-15c1e87802af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:57.742278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-e3f53ff5-7262-49cb-bec8-15c1e87802af None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:29:57.743552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e3f53ff5-7262-49cb-bec8-15c1e87802af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:57.802719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1322/2632] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:29:57.811361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba01d8e-2d28-4a1a-bef9-820e52f53d20 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:57.811647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba01d8e-2d28-4a1a-bef9-820e52f53d20 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:57.811923 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba01d8e-2d28-4a1a-bef9-820e52f53d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:57.833825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-4ba01d8e-2d28-4a1a-bef9-820e52f53d20 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:29:57.835126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-4ba01d8e-2d28-4a1a-bef9-820e52f53d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:57.843188 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1311/2633] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:29:57.847464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f53ff5-7262-49cb-bec8-15c1e87802af None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:57.857744 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f53ff5-7262-49cb-bec8-15c1e87802af None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:57.862795 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:57.863399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:57.863677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:57.863945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:57.864592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:29:57.864890 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:57.866495 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:57.878950 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1323/2634] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:29:57.883206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba01d8e-2d28-4a1a-bef9-820e52f53d20 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:57.892647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba01d8e-2d28-4a1a-bef9-820e52f53d20 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:57.897427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f06dd-b052-421f-abd2-d3a6aa85cd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:57.898017 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f06dd-b052-421f-abd2-d3a6aa85cd1b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:57.898294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f06dd-b052-421f-abd2-d3a6aa85cd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:57.898597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f06dd-b052-421f-abd2-d3a6aa85cd1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:57.899219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f06dd-b052-421f-abd2-d3a6aa85cd1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:57.901165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f06dd-b052-421f-abd2-d3a6aa85cd1b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:57.981477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1312/2635] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:29:57 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:29:57.985585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:57.996397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5ddcb-f389-441e-be05-87855b34e4a6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:58.022390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:58.057000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c020f-d99b-40b8-8ec9-48cdf2d55ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:58.057707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c020f-d99b-40b8-8ec9-48cdf2d55ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.057707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c020f-d99b-40b8-8ec9-48cdf2d55ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.058316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c020f-d99b-40b8-8ec9-48cdf2d55ca0 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.059060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c020f-d99b-40b8-8ec9-48cdf2d55ca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:58.060867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c020f-d99b-40b8-8ec9-48cdf2d55ca0 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:58.157463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1324/2636] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:29:57 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:29:58.161623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f06dd-b052-421f-abd2-d3a6aa85cd1b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:58.172650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f06dd-b052-421f-abd2-d3a6aa85cd1b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:58.177292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568b9a-8d68-4f2a-a1bb-f2ff43f7db6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:58.177884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568b9a-8d68-4f2a-a1bb-f2ff43f7db6a admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.178159 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568b9a-8d68-4f2a-a1bb-f2ff43f7db6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.178464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25568b9a-8d68-4f2a-a1bb-f2ff43f7db6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.179125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25568b9a-8d68-4f2a-a1bb-f2ff43f7db6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:58.180749 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25568b9a-8d68-4f2a-a1bb-f2ff43f7db6a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:58.190915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1313/2637] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:29:58.194823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c020f-d99b-40b8-8ec9-48cdf2d55ca0 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:58.204766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c020f-d99b-40b8-8ec9-48cdf2d55ca0 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:58.209298 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdb498-903e-4bc2-938c-9e715c5f0576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:58.209882 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb498-903e-4bc2-938c-9e715c5f0576 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.210149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb498-903e-4bc2-938c-9e715c5f0576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.210451 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb498-903e-4bc2-938c-9e715c5f0576 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.211072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdb498-903e-4bc2-938c-9e715c5f0576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:58.213185 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bdb498-903e-4bc2-938c-9e715c5f0576 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:58.222942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1325/2638] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:29:58.226968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568b9a-8d68-4f2a-a1bb-f2ff43f7db6a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:58.237743 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25568b9a-8d68-4f2a-a1bb-f2ff43f7db6a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:58.242312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0df97-c02a-4144-9c4d-df963c5a3973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:58.242963 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0df97-c02a-4144-9c4d-df963c5a3973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.243236 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0df97-c02a-4144-9c4d-df963c5a3973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.243502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0df97-c02a-4144-9c4d-df963c5a3973 admin admin] PATH_INFO: `/v3/projects/a791bea4c70d4dd0aa2ae17e4e13561f/users/5c51966de6dd4187a8c269967cecc7e7/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.258035 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0df97-c02a-4144-9c4d-df963c5a3973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a791bea4c70d4dd0aa2ae17e4e13561f, user_id=5c51966de6dd4187a8c269967cecc7e7, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:58.260690 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0df97-c02a-4144-9c4d-df963c5a3973 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:58.328021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1314/2639] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:29:58 2022] PUT /identity/v3/projects/a791bea4c70d4dd0aa2ae17e4e13561f/users/5c51966de6dd4187a8c269967cecc7e7/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:29:58.341809 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b86970-fee2-46f5-a6a6-e3d2aea5f403 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.342089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b86970-fee2-46f5-a6a6-e3d2aea5f403 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.342393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b86970-fee2-46f5-a6a6-e3d2aea5f403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.378792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c9b86970-fee2-46f5-a6a6-e3d2aea5f403 None None] MFA Rules not processed for user `5c51966de6dd4187a8c269967cecc7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:29:58.379571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c9b86970-fee2-46f5-a6a6-e3d2aea5f403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:58.446406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1326/2640] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:29:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:29:58.451198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0df97-c02a-4144-9c4d-df963c5a3973 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:58.511700 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0df97-c02a-4144-9c4d-df963c5a3973 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:58.516250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2682e1c-8cc5-4e4f-b69e-e1f429a9179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:58.516839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2682e1c-8cc5-4e4f-b69e-e1f429a9179d admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.517112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2682e1c-8cc5-4e4f-b69e-e1f429a9179d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.517377 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2682e1c-8cc5-4e4f-b69e-e1f429a9179d admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.518020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2682e1c-8cc5-4e4f-b69e-e1f429a9179d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:58.519907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2682e1c-8cc5-4e4f-b69e-e1f429a9179d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:58.615860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1315/2641] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:29:58 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:29:58.619299 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b86970-fee2-46f5-a6a6-e3d2aea5f403 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:58.630401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b86970-fee2-46f5-a6a6-e3d2aea5f403 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:58.697102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85762b-111f-4755-88be-cd5297786b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:58.697749 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85762b-111f-4755-88be-cd5297786b71 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.698066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85762b-111f-4755-88be-cd5297786b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.698358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85762b-111f-4755-88be-cd5297786b71 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.699013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85762b-111f-4755-88be-cd5297786b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:58.700919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85762b-111f-4755-88be-cd5297786b71 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:58.797712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1327/2642] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:29:58 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:29:58.802599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2682e1c-8cc5-4e4f-b69e-e1f429a9179d admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:58.813521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2682e1c-8cc5-4e4f-b69e-e1f429a9179d admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:58.818316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd66082-f33b-45d1-b1b4-61648b70e019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:58.818939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd66082-f33b-45d1-b1b4-61648b70e019 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.819219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd66082-f33b-45d1-b1b4-61648b70e019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.819492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd66082-f33b-45d1-b1b4-61648b70e019 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.820123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd66082-f33b-45d1-b1b4-61648b70e019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:58.821697 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd66082-f33b-45d1-b1b4-61648b70e019 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:58.832452 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1316/2643] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:29:58.836563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85762b-111f-4755-88be-cd5297786b71 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:58.847528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85762b-111f-4755-88be-cd5297786b71 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:58.852130 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781f388-729c-450b-8f6a-591b1840870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:58.852763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781f388-729c-450b-8f6a-591b1840870d admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:58.853056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781f388-729c-450b-8f6a-591b1840870d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:58.854430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781f388-729c-450b-8f6a-591b1840870d admin admin] PATH_INFO: `/v3/projects/3184ec8d5a244e6fb130654c7e2c1cc5/users/039aebb8b8e445138ddcba5f46f2fc65/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:58.870721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781f388-729c-450b-8f6a-591b1840870d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3184ec8d5a244e6fb130654c7e2c1cc5, user_id=039aebb8b8e445138ddcba5f46f2fc65, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:58.873352 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781f388-729c-450b-8f6a-591b1840870d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:58.952690 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1328/2644] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:29:58 2022] PUT /identity/v3/projects/3184ec8d5a244e6fb130654c7e2c1cc5/users/039aebb8b8e445138ddcba5f46f2fc65/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:29:58.956725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd66082-f33b-45d1-b1b4-61648b70e019 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:58.968094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd66082-f33b-45d1-b1b4-61648b70e019 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:59.026619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91540ef9-a2cc-4909-8920-6e18fb12d883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:59.027215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91540ef9-a2cc-4909-8920-6e18fb12d883 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:59.027492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91540ef9-a2cc-4909-8920-6e18fb12d883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:59.027757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91540ef9-a2cc-4909-8920-6e18fb12d883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:59.028409 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91540ef9-a2cc-4909-8920-6e18fb12d883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:59.029984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91540ef9-a2cc-4909-8920-6e18fb12d883 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:59.040962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1317/2645] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:29:59.045602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781f388-729c-450b-8f6a-591b1840870d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:59.051122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:59.056925 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781f388-729c-450b-8f6a-591b1840870d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:59.062225 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:59.064631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:59.064631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:59.064631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] PATH_INFO: `/v3/projects/3184ec8d5a244e6fb130654c7e2c1cc5/users/039aebb8b8e445138ddcba5f46f2fc65/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:59.065291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3184ec8d5a244e6fb130654c7e2c1cc5, user_id=039aebb8b8e445138ddcba5f46f2fc65, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:59.067934 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:59.080009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1329/2646] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:29:59 2022] PUT /identity/v3/projects/3184ec8d5a244e6fb130654c7e2c1cc5/users/039aebb8b8e445138ddcba5f46f2fc65/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:29:59.092277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b426-442d-42ea-935c-8c725473abb3 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:59.092564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b426-442d-42ea-935c-8c725473abb3 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:59.092841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183b426-442d-42ea-935c-8c725473abb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:59.099909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:29:59.123701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d183b426-442d-42ea-935c-8c725473abb3 None None] MFA Rules not processed for user `039aebb8b8e445138ddcba5f46f2fc65`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:29:59.124901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d183b426-442d-42ea-935c-8c725473abb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:59.192709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1318/2647] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:29:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:29:59.210224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:59.210224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:29:59.283064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330282ae-54d3-43a5-9bb7-09432c24ebb8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:59.286703 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1709a85-4b07-4779-9319-1562215760ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:59.287292 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709a85-4b07-4779-9319-1562215760ac service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:59.287582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709a85-4b07-4779-9319-1562215760ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:59.287830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1709a85-4b07-4779-9319-1562215760ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:59.297118 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709a85-4b07-4779-9319-1562215760ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:59.298530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1709a85-4b07-4779-9319-1562215760ac service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:59.308256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1330/2648] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:29:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:29:59.536575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183b426-442d-42ea-935c-8c725473abb3 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:29:59.546961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183b426-442d-42ea-935c-8c725473abb3 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:29:59.550594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed421b-362f-478a-ad84-adacebbd28eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:29:59.551187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed421b-362f-478a-ad84-adacebbd28eb service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:29:59.551475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed421b-362f-478a-ad84-adacebbd28eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:29:59.551727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed421b-362f-478a-ad84-adacebbd28eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:29:59.607091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed421b-362f-478a-ad84-adacebbd28eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:29:59.608602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed421b-362f-478a-ad84-adacebbd28eb service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:29:59.617703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1319/2649] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:29:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:30:04.512373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d83e2-a9e0-490a-b9dd-e57d1996f86a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:04.512984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d83e2-a9e0-490a-b9dd-e57d1996f86a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:04.512984 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d83e2-a9e0-490a-b9dd-e57d1996f86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:04.532012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-0e9d83e2-a9e0-490a-b9dd-e57d1996f86a None None] MFA Rules not processed for user `592da799f0c540d68a2cc6d54006467c`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:04.533131 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0e9d83e2-a9e0-490a-b9dd-e57d1996f86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:04.586960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1331/2650] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:09.613716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=61 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:14.580306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=64,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:48.285862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed421b-362f-478a-ad84-adacebbd28eb service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:48.354637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed421b-362f-478a-ad84-adacebbd28eb service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:48.359197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:48.359907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:48.360207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:48.360479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] PATH_INFO: `/v3/users/5c51966de6dd4187a8c269967cecc7e7` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:48.361872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c51966de6dd4187a8c269967cecc7e7)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:48.363834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:48.563836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c51966de6dd4187a8c269967cecc7e7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:48.661768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c51966de6dd4187a8c269967cecc7e7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:48.662474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c51966de6dd4187a8c269967cecc7e7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:48.671116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c51966de6dd4187a8c269967cecc7e7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:48.684267 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c51966de6dd4187a8c269967cecc7e7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:48.690444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1320/2651] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:30:48 2022] DELETE /identity/v3/users/5c51966de6dd4187a8c269967cecc7e7 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:48.695214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a352456-5a88-49fc-8923-abc5fc200fd7 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:48.695712 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a352456-5a88-49fc-8923-abc5fc200fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:48.695820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a352456-5a88-49fc-8923-abc5fc200fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:48.715833 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2a352456-5a88-49fc-8923-abc5fc200fd7 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:48.717071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2a352456-5a88-49fc-8923-abc5fc200fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:48.781517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1332/2652] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:48.791959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:48.793358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:48.852784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e440d-fce4-4911-9ec1-2a7e4441e246 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:48.856532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efad68e-6129-45c6-98f1-b0913d4d90b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:48.857145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efad68e-6129-45c6-98f1-b0913d4d90b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:48.857418 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efad68e-6129-45c6-98f1-b0913d4d90b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:48.857688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efad68e-6129-45c6-98f1-b0913d4d90b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:48.867479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efad68e-6129-45c6-98f1-b0913d4d90b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:48.868908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efad68e-6129-45c6-98f1-b0913d4d90b9 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:48.878740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1321/2653] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:30:49.335480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a352456-5a88-49fc-8923-abc5fc200fd7 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:49.346483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2a352456-5a88-49fc-8923-abc5fc200fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:49.349416 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a352456-5a88-49fc-8923-abc5fc200fd7 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:49.354593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:49.355232 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:49.355512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:49.355778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] PATH_INFO: `/v3/projects/a791bea4c70d4dd0aa2ae17e4e13561f` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:49.357817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a791bea4c70d4dd0aa2ae17e4e13561f)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:49.359274 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:49.484597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a791bea4c70d4dd0aa2ae17e4e13561f', 'request_id': 'req-67baeafa-d2cc-4b30-9e7b-5fdf89300457'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.536119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a791bea4c70d4dd0aa2ae17e4e13561f', 'request_id': 'req-67baeafa-d2cc-4b30-9e7b-5fdf89300457'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.543133 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a791bea4c70d4dd0aa2ae17e4e13561f', 'request_id': 'req-67baeafa-d2cc-4b30-9e7b-5fdf89300457'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.555186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] The token cache is being invalidate because project a791bea4c70d4dd0aa2ae17e4e13561f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:30:49.555508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.556114 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.557868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1333/2654] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:30:49 2022] DELETE /identity/v3/projects/a791bea4c70d4dd0aa2ae17e4e13561f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:49.561785 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efad68e-6129-45c6-98f1-b0913d4d90b9 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:49.573109 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efad68e-6129-45c6-98f1-b0913d4d90b9 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:49.592596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:49.633766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:49.634440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:49.634722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:49.634997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] PATH_INFO: `/v3/users/039aebb8b8e445138ddcba5f46f2fc65` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:49.636161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=039aebb8b8e445138ddcba5f46f2fc65)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:49.638091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:49.840619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '039aebb8b8e445138ddcba5f46f2fc65'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.916277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '039aebb8b8e445138ddcba5f46f2fc65'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.916914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '039aebb8b8e445138ddcba5f46f2fc65'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.925261 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '039aebb8b8e445138ddcba5f46f2fc65'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.941332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-f4a97396-0c39-4104-979e-d1bcb304ca64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '039aebb8b8e445138ddcba5f46f2fc65'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:49.947573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1322/2655] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:30:49 2022] DELETE /identity/v3/users/039aebb8b8e445138ddcba5f46f2fc65 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:50.367067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:50.368550 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:50.382820 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67baeafa-d2cc-4b30-9e7b-5fdf89300457 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:50.453231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:50.454049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:50.454414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:50.454762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] PATH_INFO: `/v3/projects/3184ec8d5a244e6fb130654c7e2c1cc5` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:50.455952 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3184ec8d5a244e6fb130654c7e2c1cc5)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:50.457863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:50.558431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:50.575554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3184ec8d5a244e6fb130654c7e2c1cc5', 'request_id': 'req-3f9f03e2-6796-45d3-995f-79e2bb89bccb'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:50.661930 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3184ec8d5a244e6fb130654c7e2c1cc5', 'request_id': 'req-3f9f03e2-6796-45d3-995f-79e2bb89bccb'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:50.668343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3184ec8d5a244e6fb130654c7e2c1cc5', 'request_id': 'req-3f9f03e2-6796-45d3-995f-79e2bb89bccb'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:50.681375 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] The token cache is being invalidate because project 3184ec8d5a244e6fb130654c7e2c1cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:30:50.681659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:50.682304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3f9f03e2-6796-45d3-995f-79e2bb89bccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:50.683861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1334/2656] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:30:50 2022] DELETE /identity/v3/projects/3184ec8d5a244e6fb130654c7e2c1cc5 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:50.688891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f0207-6513-4e05-86d5-8b49313b459c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:50.689355 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f0207-6513-4e05-86d5-8b49313b459c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:50.689454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f0207-6513-4e05-86d5-8b49313b459c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:50.711908 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f37f0207-6513-4e05-86d5-8b49313b459c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:50.713091 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f37f0207-6513-4e05-86d5-8b49313b459c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:50.776736 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1323/2657] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:50.784354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf843a-2124-44e0-8250-0671c2e0e2d6 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:50.784640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf843a-2124-44e0-8250-0671c2e0e2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:50.784912 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf843a-2124-44e0-8250-0671c2e0e2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:50.806033 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-dfdf843a-2124-44e0-8250-0671c2e0e2d6 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:50.807282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-dfdf843a-2124-44e0-8250-0671c2e0e2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:50.816405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1335/2658] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:30:50.821351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f0207-6513-4e05-86d5-8b49313b459c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:50.831877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f0207-6513-4e05-86d5-8b49313b459c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:50.836968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:50.837580 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:50.837853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:50.838123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:50.838805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:30:50.839108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:50.840708 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:50.853638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1324/2659] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:30:50.858167 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf843a-2124-44e0-8250-0671c2e0e2d6 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:50.870007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf843a-2124-44e0-8250-0671c2e0e2d6 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:50.875143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820715d0-95e0-477c-a243-1b5c03ccc211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:50.875745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820715d0-95e0-477c-a243-1b5c03ccc211 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:50.876020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820715d0-95e0-477c-a243-1b5c03ccc211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:50.876290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820715d0-95e0-477c-a243-1b5c03ccc211 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:50.876918 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820715d0-95e0-477c-a243-1b5c03ccc211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:50.878866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820715d0-95e0-477c-a243-1b5c03ccc211 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:51.004174 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1336/2660] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:30:50 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:30:51.008665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:51.019285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8984c-aee6-413a-b02c-061d9cbd108b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:51.047273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:51.079033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06614e2d-ddef-489e-b4fa-eeef42b300ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:51.079653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06614e2d-ddef-489e-b4fa-eeef42b300ab admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.079928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06614e2d-ddef-489e-b4fa-eeef42b300ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.080255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06614e2d-ddef-489e-b4fa-eeef42b300ab admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.080888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06614e2d-ddef-489e-b4fa-eeef42b300ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:51.082856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06614e2d-ddef-489e-b4fa-eeef42b300ab admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:51.180124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1325/2661] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:30:51 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:30:51.184609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820715d0-95e0-477c-a243-1b5c03ccc211 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:51.196027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820715d0-95e0-477c-a243-1b5c03ccc211 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:51.201244 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25e9f7-765d-4e57-9c11-947376e6b339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:51.201900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25e9f7-765d-4e57-9c11-947376e6b339 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.202178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25e9f7-765d-4e57-9c11-947376e6b339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.202488 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25e9f7-765d-4e57-9c11-947376e6b339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.203175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25e9f7-765d-4e57-9c11-947376e6b339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:51.204941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b25e9f7-765d-4e57-9c11-947376e6b339 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:51.220898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1337/2662] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:30:51.225520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06614e2d-ddef-489e-b4fa-eeef42b300ab admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:51.236026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06614e2d-ddef-489e-b4fa-eeef42b300ab admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:51.242316 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47927d-5127-425c-9a08-d90a02cb880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:51.242962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47927d-5127-425c-9a08-d90a02cb880b admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.243243 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47927d-5127-425c-9a08-d90a02cb880b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.243515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47927d-5127-425c-9a08-d90a02cb880b admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.244210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47927d-5127-425c-9a08-d90a02cb880b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:51.245916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47927d-5127-425c-9a08-d90a02cb880b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:51.260494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1326/2663] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:30:51.261640 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25e9f7-765d-4e57-9c11-947376e6b339 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:51.272496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25e9f7-765d-4e57-9c11-947376e6b339 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:51.277812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae30c3-cfca-45f1-a658-36cbebdd5da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:51.278487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae30c3-cfca-45f1-a658-36cbebdd5da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.278761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae30c3-cfca-45f1-a658-36cbebdd5da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.279029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae30c3-cfca-45f1-a658-36cbebdd5da6 admin admin] PATH_INFO: `/v3/projects/523810f32e1e4446a4444601f9b27827/users/de7a43296fcc4c25898f610948566793/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.295861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae30c3-cfca-45f1-a658-36cbebdd5da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523810f32e1e4446a4444601f9b27827, user_id=de7a43296fcc4c25898f610948566793, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:51.298560 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beae30c3-cfca-45f1-a658-36cbebdd5da6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:51.369658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1338/2664] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:30:51 2022] PUT /identity/v3/projects/523810f32e1e4446a4444601f9b27827/users/de7a43296fcc4c25898f610948566793/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:30:51.380179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf4f31-a500-4145-a838-f94cd94b4b3b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.380460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf4f31-a500-4145-a838-f94cd94b4b3b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.380733 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf4f31-a500-4145-a838-f94cd94b4b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.412709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-dfcf4f31-a500-4145-a838-f94cd94b4b3b None None] MFA Rules not processed for user `de7a43296fcc4c25898f610948566793`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:51.414559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dfcf4f31-a500-4145-a838-f94cd94b4b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:51.489128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1327/2665] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:51.493951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae30c3-cfca-45f1-a658-36cbebdd5da6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:51.559736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae30c3-cfca-45f1-a658-36cbebdd5da6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:51.564674 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7193-4d19-4fad-b671-3d28e4048485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:51.565278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7193-4d19-4fad-b671-3d28e4048485 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.565554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7193-4d19-4fad-b671-3d28e4048485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.565822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7193-4d19-4fad-b671-3d28e4048485 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.566544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f7193-4d19-4fad-b671-3d28e4048485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:51.568435 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f7193-4d19-4fad-b671-3d28e4048485 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:51.707026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1339/2666] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:30:51 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:30:51.708179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf4f31-a500-4145-a838-f94cd94b4b3b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:51.720454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf4f31-a500-4145-a838-f94cd94b4b3b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:51.783069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968fe8-3e39-4045-9466-cab885a69a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:51.783671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968fe8-3e39-4045-9466-cab885a69a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.783954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968fe8-3e39-4045-9466-cab885a69a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.784224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae968fe8-3e39-4045-9466-cab885a69a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.784870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae968fe8-3e39-4045-9466-cab885a69a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:51.786881 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae968fe8-3e39-4045-9466-cab885a69a5e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:51.862676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1328/2667] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:30:51 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:30:51.867253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7193-4d19-4fad-b671-3d28e4048485 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:51.877729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7193-4d19-4fad-b671-3d28e4048485 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:51.883496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608008c9-855b-4620-9ae3-08526ce4ed65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:51.884230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608008c9-855b-4620-9ae3-08526ce4ed65 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.884511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608008c9-855b-4620-9ae3-08526ce4ed65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.884808 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608008c9-855b-4620-9ae3-08526ce4ed65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.885443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608008c9-855b-4620-9ae3-08526ce4ed65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:51.887080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608008c9-855b-4620-9ae3-08526ce4ed65 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:51.899329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1340/2668] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:30:51.903160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968fe8-3e39-4045-9466-cab885a69a5e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:51.914900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae968fe8-3e39-4045-9466-cab885a69a5e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:51.920431 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5413d3-5545-4074-937f-3c2b96391e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:51.921086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5413d3-5545-4074-937f-3c2b96391e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:51.921367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5413d3-5545-4074-937f-3c2b96391e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:51.921637 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5413d3-5545-4074-937f-3c2b96391e46 admin admin] PATH_INFO: `/v3/projects/c1da54f231304b1bb2011528b9ef4202/users/2bfcbe27148b4c379fff0ae2a2fdd7ab/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:51.937825 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5413d3-5545-4074-937f-3c2b96391e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1da54f231304b1bb2011528b9ef4202, user_id=2bfcbe27148b4c379fff0ae2a2fdd7ab, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:51.940612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5413d3-5545-4074-937f-3c2b96391e46 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:52.019827 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1329/2669] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:30:51 2022] PUT /identity/v3/projects/c1da54f231304b1bb2011528b9ef4202/users/2bfcbe27148b4c379fff0ae2a2fdd7ab/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:52.024427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608008c9-855b-4620-9ae3-08526ce4ed65 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:52.034964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608008c9-855b-4620-9ae3-08526ce4ed65 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:52.055704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:52.095020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e4ece-80b9-4153-abf1-80fd4e802aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:52.095733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4ece-80b9-4153-abf1-80fd4e802aab admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:52.096082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4ece-80b9-4153-abf1-80fd4e802aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:52.096403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4ece-80b9-4153-abf1-80fd4e802aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:52.097043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e4ece-80b9-4153-abf1-80fd4e802aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:52.098653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e4ece-80b9-4153-abf1-80fd4e802aab admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:52.108946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1341/2670] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:30:52.113468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5413d3-5545-4074-937f-3c2b96391e46 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:52.120425 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:52.126935 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5413d3-5545-4074-937f-3c2b96391e46 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:52.132928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:52.133230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:52.133725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:52.134174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] PATH_INFO: `/v3/projects/c1da54f231304b1bb2011528b9ef4202/users/2bfcbe27148b4c379fff0ae2a2fdd7ab/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:52.136183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1da54f231304b1bb2011528b9ef4202, user_id=2bfcbe27148b4c379fff0ae2a2fdd7ab, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:52.138951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:52.149860 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1330/2671] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:30:52 2022] PUT /identity/v3/projects/c1da54f231304b1bb2011528b9ef4202/users/2bfcbe27148b4c379fff0ae2a2fdd7ab/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:30:52.161502 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf386a-f53b-4977-a9dc-e301c1012264 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:52.161798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf386a-f53b-4977-a9dc-e301c1012264 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:52.162992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf386a-f53b-4977-a9dc-e301c1012264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:52.191653 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-9ecf386a-f53b-4977-a9dc-e301c1012264 None None] MFA Rules not processed for user `2bfcbe27148b4c379fff0ae2a2fdd7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:52.192891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9ecf386a-f53b-4977-a9dc-e301c1012264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:52.263449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1342/2672] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:52.280213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:52.281918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:52.345608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7b667-e208-4bbc-b5e2-59be8851af3a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:52.349585 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad585a6a-132e-46d3-b677-50ab8726dc13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:52.350186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad585a6a-132e-46d3-b677-50ab8726dc13 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:52.350502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad585a6a-132e-46d3-b677-50ab8726dc13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:52.350767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad585a6a-132e-46d3-b677-50ab8726dc13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:52.360715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad585a6a-132e-46d3-b677-50ab8726dc13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:52.362427 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad585a6a-132e-46d3-b677-50ab8726dc13 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:52.372018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1331/2673] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:30:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:30:53.369985 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf386a-f53b-4977-a9dc-e301c1012264 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:53.379909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf386a-f53b-4977-a9dc-e301c1012264 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:53.385633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bfcbe27148b4c379fff0ae2a2fdd7ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1da54f231304b1bb2011528b9ef4202', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:53.386750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:53.386750 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:53.386951 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:53.387429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:30:53.387732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:53.389316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:53.402069 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1343/2674] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:30:53.414698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad585a6a-132e-46d3-b677-50ab8726dc13 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:53.419040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad585a6a-132e-46d3-b677-50ab8726dc13 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:53.437177 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d848a46-8417-4c2f-9ba5-a69bbcc43e80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bfcbe27148b4c379fff0ae2a2fdd7ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1da54f231304b1bb2011528b9ef4202', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:53.438281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d848a46-8417-4c2f-9ba5-a69bbcc43e80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:53.438835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d848a46-8417-4c2f-9ba5-a69bbcc43e80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:53.439344 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d848a46-8417-4c2f-9ba5-a69bbcc43e80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:53.440569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d848a46-8417-4c2f-9ba5-a69bbcc43e80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:53.446426 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d848a46-8417-4c2f-9ba5-a69bbcc43e80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:53.526980 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1332/2675] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:30:53 2022] POST /identity/v3/projects => generated 354 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:30:53.531068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:53.541764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc0625-aa90-4193-b1f5-24ad7b4f4953 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:53.599647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bfcbe27148b4c379fff0ae2a2fdd7ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1da54f231304b1bb2011528b9ef4202', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:53.600259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:53.600541 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:53.600819 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:53.602253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:30:53.602588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:53.604211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:53.618265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1344/2676] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:30:53.659416 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d848a46-8417-4c2f-9ba5-a69bbcc43e80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:53.670938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d848a46-8417-4c2f-9ba5-a69bbcc43e80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:53.675909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2bfcbe27148b4c379fff0ae2a2fdd7ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1da54f231304b1bb2011528b9ef4202', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:53.676606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:53.676959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:53.677327 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] PATH_INFO: `/v3/projects/b73dff51a64343cb8970821f91441350` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:53.678587 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b73dff51a64343cb8970821f91441350)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:53.681150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:53.807213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73dff51a64343cb8970821f91441350', 'request_id': 'req-a9cfed29-9561-49ab-be86-c3bc20943f80'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:53.878636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73dff51a64343cb8970821f91441350', 'request_id': 'req-a9cfed29-9561-49ab-be86-c3bc20943f80'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:53.886554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73dff51a64343cb8970821f91441350', 'request_id': 'req-a9cfed29-9561-49ab-be86-c3bc20943f80'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:53.900198 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] The token cache is being invalidate because project b73dff51a64343cb8970821f91441350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:30:53.900957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:53.901183 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9cfed29-9561-49ab-be86-c3bc20943f80 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:53.906429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1333/2677] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:30:53 2022] DELETE /identity/v3/projects/b73dff51a64343cb8970821f91441350 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:54.568430 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:54.571720 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:54.604334 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:54.638149 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cc31e-27ba-4796-a489-f2f8f0d41aa4 tempest-VolumeQuotaClassesTest-2013772346 tempest-VolumeQuotaClassesTest-2013772346-project-admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:54.697151 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:54.697778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:54.698042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:54.698311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] PATH_INFO: `/v3/users/de7a43296fcc4c25898f610948566793` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:54.699450 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7a43296fcc4c25898f610948566793)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:54.701459 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:54.946035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7a43296fcc4c25898f610948566793'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:54.949704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7a43296fcc4c25898f610948566793'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:55.065559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7a43296fcc4c25898f610948566793'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:55.084957 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7a43296fcc4c25898f610948566793'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:55.098531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7a43296fcc4c25898f610948566793'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:55.110262 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1345/2678] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:30:54 2022] DELETE /identity/v3/users/de7a43296fcc4c25898f610948566793 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:55.117723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2949de-0f38-4209-bc54-e138c4e35c66 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:55.117723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2949de-0f38-4209-bc54-e138c4e35c66 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:55.121059 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2949de-0f38-4209-bc54-e138c4e35c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:55.155836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5e2949de-0f38-4209-bc54-e138c4e35c66 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:55.157813 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5e2949de-0f38-4209-bc54-e138c4e35c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:55.243683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:55.265828 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1334/2679] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:30:55.277817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:55.279875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:55.343442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c14f4f-9200-469c-afa2-47f9d19879a2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:55.346998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9bd42-ca84-48a3-93b3-445d07452ea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:55.347601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9bd42-ca84-48a3-93b3-445d07452ea7 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:55.347876 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9bd42-ca84-48a3-93b3-445d07452ea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:55.348145 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9bd42-ca84-48a3-93b3-445d07452ea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:55.357537 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:55.360202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9bd42-ca84-48a3-93b3-445d07452ea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:55.361631 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9bd42-ca84-48a3-93b3-445d07452ea7 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:55.373045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1346/2680] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:30:55.878419 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2949de-0f38-4209-bc54-e138c4e35c66 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:55.879957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5e2949de-0f38-4209-bc54-e138c4e35c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:55.893484 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2949de-0f38-4209-bc54-e138c4e35c66 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:55.898739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:55.899375 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:55.899653 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:55.899907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] PATH_INFO: `/v3/projects/523810f32e1e4446a4444601f9b27827` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:55.901033 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] RBAC: Authorizing `identity:delete_project(project_id=523810f32e1e4446a4444601f9b27827)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:55.902983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:56.016685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523810f32e1e4446a4444601f9b27827', 'request_id': 'req-c61e6933-955f-48a1-93d2-127fd7f9623c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.101450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523810f32e1e4446a4444601f9b27827', 'request_id': 'req-c61e6933-955f-48a1-93d2-127fd7f9623c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.107420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523810f32e1e4446a4444601f9b27827', 'request_id': 'req-c61e6933-955f-48a1-93d2-127fd7f9623c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.119691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] The token cache is being invalidate because project 523810f32e1e4446a4444601f9b27827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:30:56.120282 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.120584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.122082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1335/2681] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:30:55 2022] DELETE /identity/v3/projects/523810f32e1e4446a4444601f9b27827 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:56.126064 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9bd42-ca84-48a3-93b3-445d07452ea7 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:56.136467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9bd42-ca84-48a3-93b3-445d07452ea7 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:56.193399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:56.194146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:56.194453 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:56.195384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] PATH_INFO: `/v3/users/2bfcbe27148b4c379fff0ae2a2fdd7ab` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:56.196510 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bfcbe27148b4c379fff0ae2a2fdd7ab)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:56.198499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:56.342770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bfcbe27148b4c379fff0ae2a2fdd7ab'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.344342 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bfcbe27148b4c379fff0ae2a2fdd7ab'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.414236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bfcbe27148b4c379fff0ae2a2fdd7ab'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.428337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bfcbe27148b4c379fff0ae2a2fdd7ab'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.436111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4e52652b-10fb-4a40-a8d8-3a1bd114d01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bfcbe27148b4c379fff0ae2a2fdd7ab'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:56.442305 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1347/2682] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:30:56 2022] DELETE /identity/v3/users/2bfcbe27148b4c379fff0ae2a2fdd7ab => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:56.873036 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:56.874557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:56.889502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e6933-955f-48a1-93d2-127fd7f9623c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:56.950566 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:56.951199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:56.951648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:56.951926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] PATH_INFO: `/v3/projects/c1da54f231304b1bb2011528b9ef4202` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:56.953050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1da54f231304b1bb2011528b9ef4202)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:56.955052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:57.082862 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1da54f231304b1bb2011528b9ef4202', 'request_id': 'req-bf6cc3c0-90b6-40db-8665-f82a34c516fb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:57.147666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1da54f231304b1bb2011528b9ef4202', 'request_id': 'req-bf6cc3c0-90b6-40db-8665-f82a34c516fb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:57.154621 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1da54f231304b1bb2011528b9ef4202', 'request_id': 'req-bf6cc3c0-90b6-40db-8665-f82a34c516fb'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:57.167015 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] The token cache is being invalidate because project c1da54f231304b1bb2011528b9ef4202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:30:57.167305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:57.168027 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-bf6cc3c0-90b6-40db-8665-f82a34c516fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:30:57.169567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1336/2683] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:30:56 2022] DELETE /identity/v3/projects/c1da54f231304b1bb2011528b9ef4202 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:57.174791 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7cfc-9b14-4e39-aa8e-fdc17a63fd55 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.175268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7cfc-9b14-4e39-aa8e-fdc17a63fd55 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.175370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f7cfc-9b14-4e39-aa8e-fdc17a63fd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.196198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3c8f7cfc-9b14-4e39-aa8e-fdc17a63fd55 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:57.197332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3c8f7cfc-9b14-4e39-aa8e-fdc17a63fd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:57.259717 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1348/2684] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:30:57.267853 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441651b7-d07f-4d02-8e83-a8fd93c7d9ff None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.268135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441651b7-d07f-4d02-8e83-a8fd93c7d9ff None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.268394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441651b7-d07f-4d02-8e83-a8fd93c7d9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.282741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:57.288900 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-441651b7-d07f-4d02-8e83-a8fd93c7d9ff None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:57.290380 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-441651b7-d07f-4d02-8e83-a8fd93c7d9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:57.298462 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1337/2685] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:57.302589 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f7cfc-9b14-4e39-aa8e-fdc17a63fd55 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:57.313179 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f7cfc-9b14-4e39-aa8e-fdc17a63fd55 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:57.317766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:57.318371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.318661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.318926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.319879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:30:57.320183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:57.321840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:57.333542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1349/2686] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:30:57.338612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441651b7-d07f-4d02-8e83-a8fd93c7d9ff None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:57.348387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441651b7-d07f-4d02-8e83-a8fd93c7d9ff None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:57.353806 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b60e0-bf17-4665-90c8-a61ab502d98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:57.354460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b60e0-bf17-4665-90c8-a61ab502d98e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.356924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b60e0-bf17-4665-90c8-a61ab502d98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.356924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b60e0-bf17-4665-90c8-a61ab502d98e admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.356924 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b60e0-bf17-4665-90c8-a61ab502d98e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:57.357512 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b60e0-bf17-4665-90c8-a61ab502d98e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:57.479042 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1338/2687] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:30:57 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:30:57.483634 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:57.490009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:57.497177 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5924c6a-131a-4adc-b248-815d88f64ca1 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:57.578528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249dd12-39b9-4c09-a68f-27ee0d60a612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:57.579316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249dd12-39b9-4c09-a68f-27ee0d60a612 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.579768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249dd12-39b9-4c09-a68f-27ee0d60a612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.580183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249dd12-39b9-4c09-a68f-27ee0d60a612 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.580965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249dd12-39b9-4c09-a68f-27ee0d60a612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:57.583051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249dd12-39b9-4c09-a68f-27ee0d60a612 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:57.654431 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1350/2688] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:30:57 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:30:57.658912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b60e0-bf17-4665-90c8-a61ab502d98e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:57.670085 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b60e0-bf17-4665-90c8-a61ab502d98e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:57.674912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8c362-7583-494a-8516-3d5c05cdedff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:57.675515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8c362-7583-494a-8516-3d5c05cdedff admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.675790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8c362-7583-494a-8516-3d5c05cdedff admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.676067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8c362-7583-494a-8516-3d5c05cdedff admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.676715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8c362-7583-494a-8516-3d5c05cdedff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:57.678299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8c362-7583-494a-8516-3d5c05cdedff admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:57.690153 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1339/2689] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:30:57.694006 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249dd12-39b9-4c09-a68f-27ee0d60a612 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:57.704911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249dd12-39b9-4c09-a68f-27ee0d60a612 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:57.709588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a70745-f3a1-47ed-8f54-30d9e2c5891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:57.710768 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a70745-f3a1-47ed-8f54-30d9e2c5891d admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.711050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a70745-f3a1-47ed-8f54-30d9e2c5891d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.711320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a70745-f3a1-47ed-8f54-30d9e2c5891d admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.711958 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a70745-f3a1-47ed-8f54-30d9e2c5891d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:57.716372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a70745-f3a1-47ed-8f54-30d9e2c5891d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:57.729780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1351/2690] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:30:57.731022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8c362-7583-494a-8516-3d5c05cdedff admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:57.742835 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8c362-7583-494a-8516-3d5c05cdedff admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:57.748072 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f56aa-6fb1-4594-8cbf-83e0efafb6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:57.748728 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f56aa-6fb1-4594-8cbf-83e0efafb6ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.749001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f56aa-6fb1-4594-8cbf-83e0efafb6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.749274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f56aa-6fb1-4594-8cbf-83e0efafb6ba admin admin] PATH_INFO: `/v3/projects/2c85d9f01e1a49b798b8d3bd0ac573cf/users/71997f86bcdc4faf95b5d8800a78ba75/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.765764 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f56aa-6fb1-4594-8cbf-83e0efafb6ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c85d9f01e1a49b798b8d3bd0ac573cf, user_id=71997f86bcdc4faf95b5d8800a78ba75, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:57.768630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f56aa-6fb1-4594-8cbf-83e0efafb6ba admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:57.875999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1340/2691] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:30:57 2022] PUT /identity/v3/projects/2c85d9f01e1a49b798b8d3bd0ac573cf/users/71997f86bcdc4faf95b5d8800a78ba75/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:30:57.879850 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f6a81-6d25-44c2-ab68-87dec8865246 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:57.880726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f6a81-6d25-44c2-ab68-87dec8865246 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:57.881155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f6a81-6d25-44c2-ab68-87dec8865246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:57.920559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-2c0f6a81-6d25-44c2-ab68-87dec8865246 None None] MFA Rules not processed for user `71997f86bcdc4faf95b5d8800a78ba75`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:57.922528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2c0f6a81-6d25-44c2-ab68-87dec8865246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:58.002903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1352/2692] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:58.003031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f56aa-6fb1-4594-8cbf-83e0efafb6ba admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:58.067139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f56aa-6fb1-4594-8cbf-83e0efafb6ba admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:58.072514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63dd1d-df31-4330-88d3-b61670eda5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:58.073119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63dd1d-df31-4330-88d3-b61670eda5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:58.073424 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63dd1d-df31-4330-88d3-b61670eda5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:58.073748 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63dd1d-df31-4330-88d3-b61670eda5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:58.074414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63dd1d-df31-4330-88d3-b61670eda5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:58.076308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63dd1d-df31-4330-88d3-b61670eda5c3 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:58.203917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1341/2693] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:30:58 2022] POST /identity/v3/projects => generated 377 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:30:58.208112 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f6a81-6d25-44c2-ab68-87dec8865246 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:58.219776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f6a81-6d25-44c2-ab68-87dec8865246 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:58.277351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58972e4-ee81-45b4-9c90-c074d23e22cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:58.277903 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58972e4-ee81-45b4-9c90-c074d23e22cd admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:58.278180 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58972e4-ee81-45b4-9c90-c074d23e22cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:58.278513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58972e4-ee81-45b4-9c90-c074d23e22cd admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:58.279160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58972e4-ee81-45b4-9c90-c074d23e22cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:58.281041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58972e4-ee81-45b4-9c90-c074d23e22cd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:58.380374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:58.385658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1353/2694] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:30:58 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:30:58.389607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63dd1d-df31-4330-88d3-b61670eda5c3 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:58.400960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63dd1d-df31-4330-88d3-b61670eda5c3 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:58.406006 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2d398-7207-4faf-95d3-5846afbb8238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:58.406631 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2d398-7207-4faf-95d3-5846afbb8238 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:58.406909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2d398-7207-4faf-95d3-5846afbb8238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:58.407179 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2d398-7207-4faf-95d3-5846afbb8238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:58.407808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2d398-7207-4faf-95d3-5846afbb8238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:58.409411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2d398-7207-4faf-95d3-5846afbb8238 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:58.436511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1342/2695] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:30:58.440584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58972e4-ee81-45b4-9c90-c074d23e22cd admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:58.451554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58972e4-ee81-45b4-9c90-c074d23e22cd admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:58.456635 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfedaf-728d-4425-9660-6a61432b7d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:58.457721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfedaf-728d-4425-9660-6a61432b7d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:58.457721 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfedaf-728d-4425-9660-6a61432b7d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:58.457944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfedaf-728d-4425-9660-6a61432b7d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:58.458521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfedaf-728d-4425-9660-6a61432b7d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:58.460786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfedaf-728d-4425-9660-6a61432b7d4d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:58.471582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1354/2696] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:30:58.475692 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2d398-7207-4faf-95d3-5846afbb8238 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:58.487219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2d398-7207-4faf-95d3-5846afbb8238 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:58.492447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5682cbe-c09e-4af3-a0c2-ccb8b55bdfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:58.493126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5682cbe-c09e-4af3-a0c2-ccb8b55bdfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:58.493406 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5682cbe-c09e-4af3-a0c2-ccb8b55bdfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:58.493679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5682cbe-c09e-4af3-a0c2-ccb8b55bdfb1 admin admin] PATH_INFO: `/v3/projects/5dadf6be07af45e09f5f770f2e3f7f3f/users/8fd0671bf1b24df6b15cb77336b1730e/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:58.514364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5682cbe-c09e-4af3-a0c2-ccb8b55bdfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dadf6be07af45e09f5f770f2e3f7f3f, user_id=8fd0671bf1b24df6b15cb77336b1730e, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:58.517391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5682cbe-c09e-4af3-a0c2-ccb8b55bdfb1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:58.596866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1343/2697] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:30:58 2022] PUT /identity/v3/projects/5dadf6be07af45e09f5f770f2e3f7f3f/users/8fd0671bf1b24df6b15cb77336b1730e/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:58.607734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fdf3e-ada7-4835-845f-631a23d47f98 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:58.608045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fdf3e-ada7-4835-845f-631a23d47f98 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:58.608340 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fdf3e-ada7-4835-845f-631a23d47f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:58.638214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-c07fdf3e-ada7-4835-845f-631a23d47f98 None None] MFA Rules not processed for user `8fd0671bf1b24df6b15cb77336b1730e`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:58.639475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c07fdf3e-ada7-4835-845f-631a23d47f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:58.703263 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1355/2698] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:58.707820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5682cbe-c09e-4af3-a0c2-ccb8b55bdfb1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:58.762722 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:58.773415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5682cbe-c09e-4af3-a0c2-ccb8b55bdfb1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:58.778311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d26e83-e6db-4a78-a9e4-43d0e8b23b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:58.779364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d26e83-e6db-4a78-a9e4-43d0e8b23b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:58.779838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d26e83-e6db-4a78-a9e4-43d0e8b23b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:58.780132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d26e83-e6db-4a78-a9e4-43d0e8b23b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:58.780805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d26e83-e6db-4a78-a9e4-43d0e8b23b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:58.782746 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d26e83-e6db-4a78-a9e4-43d0e8b23b0b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:58.891967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1344/2699] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:30:58 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:30:58.893152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fdf3e-ada7-4835-845f-631a23d47f98 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:58.905637 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fdf3e-ada7-4835-845f-631a23d47f98 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:58.968761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007654-a4a8-48dd-baef-3b71b2cbd025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:58.969367 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007654-a4a8-48dd-baef-3b71b2cbd025 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:58.969642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007654-a4a8-48dd-baef-3b71b2cbd025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:58.969908 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007654-a4a8-48dd-baef-3b71b2cbd025 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:58.970574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5007654-a4a8-48dd-baef-3b71b2cbd025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:58.972454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5007654-a4a8-48dd-baef-3b71b2cbd025 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:59.160401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1356/2700] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:30:58 2022] POST /identity/v3/users => generated 295 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:30:59.172947 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d26e83-e6db-4a78-a9e4-43d0e8b23b0b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:59.184185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d26e83-e6db-4a78-a9e4-43d0e8b23b0b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:59.189386 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129a6a8-9df0-47a5-8325-0e590a887650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:59.193084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a6a8-9df0-47a5-8325-0e590a887650 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:59.193084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a6a8-9df0-47a5-8325-0e590a887650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:59.193084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129a6a8-9df0-47a5-8325-0e590a887650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:59.193084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129a6a8-9df0-47a5-8325-0e590a887650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:59.193084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129a6a8-9df0-47a5-8325-0e590a887650 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:59.208116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1345/2701] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:30:59.212807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007654-a4a8-48dd-baef-3b71b2cbd025 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:59.223479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007654-a4a8-48dd-baef-3b71b2cbd025 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:59.229048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ab413-b4fc-4c97-bbd9-e9a6f2584002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:59.229898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ab413-b4fc-4c97-bbd9-e9a6f2584002 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:59.230176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ab413-b4fc-4c97-bbd9-e9a6f2584002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:59.230480 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ab413-b4fc-4c97-bbd9-e9a6f2584002 admin admin] PATH_INFO: `/v3/projects/414f8b55025e40bc9f65b0e795106ad9/users/07f7e5607f554b959c7155dee1296e1d/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:59.247429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ab413-b4fc-4c97-bbd9-e9a6f2584002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414f8b55025e40bc9f65b0e795106ad9, user_id=07f7e5607f554b959c7155dee1296e1d, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:59.249333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ab413-b4fc-4c97-bbd9-e9a6f2584002 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:59.340629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1357/2702] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:30:59 2022] PUT /identity/v3/projects/414f8b55025e40bc9f65b0e795106ad9/users/07f7e5607f554b959c7155dee1296e1d/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:59.344771 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129a6a8-9df0-47a5-8325-0e590a887650 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:59.355295 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129a6a8-9df0-47a5-8325-0e590a887650 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:59.419358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969b3b5d-b195-4397-9952-62c63d8a21f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:59.419866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b3b5d-b195-4397-9952-62c63d8a21f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:59.420144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b3b5d-b195-4397-9952-62c63d8a21f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:59.420546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969b3b5d-b195-4397-9952-62c63d8a21f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:59.421192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b3b5d-b195-4397-9952-62c63d8a21f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:59.422948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969b3b5d-b195-4397-9952-62c63d8a21f1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:59.434215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1346/2703] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:30:59.438447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ab413-b4fc-4c97-bbd9-e9a6f2584002 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:59.449579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ab413-b4fc-4c97-bbd9-e9a6f2584002 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:59.454159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:59.454835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:59.455113 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:59.455384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] PATH_INFO: `/v3/projects/414f8b55025e40bc9f65b0e795106ad9/users/07f7e5607f554b959c7155dee1296e1d/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:59.457304 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414f8b55025e40bc9f65b0e795106ad9, user_id=07f7e5607f554b959c7155dee1296e1d, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:59.460009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:59.471310 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1358/2704] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:30:59 2022] PUT /identity/v3/projects/414f8b55025e40bc9f65b0e795106ad9/users/07f7e5607f554b959c7155dee1296e1d/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:30:59.482119 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce365bc0-64e8-4ece-8ae7-ca0e2cae4c57 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:59.482435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce365bc0-64e8-4ece-8ae7-ca0e2cae4c57 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:59.482739 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce365bc0-64e8-4ece-8ae7-ca0e2cae4c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:59.511950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-ce365bc0-64e8-4ece-8ae7-ca0e2cae4c57 None None] MFA Rules not processed for user `07f7e5607f554b959c7155dee1296e1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:30:59.513187 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ce365bc0-64e8-4ece-8ae7-ca0e2cae4c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:59.579945 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1347/2705] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:30:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:30:59.591761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:59.593415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:59.627408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:59.650584 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4d721-dbe2-4f97-b3ea-a29667fa6171 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:59.654253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:59.654872 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:59.655144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:59.655410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:59.664390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:59.665805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:59.675505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1359/2706] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:30:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:30:59.791370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce365bc0-64e8-4ece-8ae7-ca0e2cae4c57 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:30:59.798129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ce365bc0-64e8-4ece-8ae7-ca0e2cae4c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:30:59.805233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce365bc0-64e8-4ece-8ae7-ca0e2cae4c57 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:30:59.808811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea923a-c76e-4769-aff4-ecdf9abc223a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:30:59.809405 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea923a-c76e-4769-aff4-ecdf9abc223a service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:30:59.809682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea923a-c76e-4769-aff4-ecdf9abc223a service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:30:59.809949 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea923a-c76e-4769-aff4-ecdf9abc223a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:30:59.856242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:30:59.869214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea923a-c76e-4769-aff4-ecdf9abc223a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:30:59.870680 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea923a-c76e-4769-aff4-ecdf9abc223a service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:30:59.879413 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1348/2707] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:30:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:00.487358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:00.488779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:00.547257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43c274-893b-48b8-b0d8-c4484339d6cd service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:00.550838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc874-6cd5-405d-a0fb-ef2266b8a065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:00.551429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc874-6cd5-405d-a0fb-ef2266b8a065 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:00.551699 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc874-6cd5-405d-a0fb-ef2266b8a065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:00.551964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc874-6cd5-405d-a0fb-ef2266b8a065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:00.553675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-bcfdc874-6cd5-405d-a0fb-ef2266b8a065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:00.616798 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdc874-6cd5-405d-a0fb-ef2266b8a065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:00.618152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdc874-6cd5-405d-a0fb-ef2266b8a065 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:00.627606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1360/2708] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:04.115210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea923a-c76e-4769-aff4-ecdf9abc223a service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:04.129311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea923a-c76e-4769-aff4-ecdf9abc223a service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:04.138474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:04.138474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:04.138474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:04.138474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] PATH_INFO: `/v3/users/71997f86bcdc4faf95b5d8800a78ba75` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:04.139058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71997f86bcdc4faf95b5d8800a78ba75)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:04.140672 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:04.423165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71997f86bcdc4faf95b5d8800a78ba75'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:04.508123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71997f86bcdc4faf95b5d8800a78ba75'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:04.508738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71997f86bcdc4faf95b5d8800a78ba75'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:04.516453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71997f86bcdc4faf95b5d8800a78ba75'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:04.530681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71997f86bcdc4faf95b5d8800a78ba75'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:04.537354 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1349/2709] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:04 2022] DELETE /identity/v3/users/71997f86bcdc4faf95b5d8800a78ba75 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:04.542115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f4f64-f0a9-44f9-a61a-064c0b0fefd6 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:04.542671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f4f64-f0a9-44f9-a61a-064c0b0fefd6 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:04.542671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f4f64-f0a9-44f9-a61a-064c0b0fefd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:04.563501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-964f4f64-f0a9-44f9-a61a-064c0b0fefd6 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:04.564757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-964f4f64-f0a9-44f9-a61a-064c0b0fefd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:04.627711 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1361/2710] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:04.638361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:04.639847 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:04.702050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e8e2b-b69f-4e40-8205-629cb0cfd710 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:04.705647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a88c4-829a-44de-a14b-7e37ffd757e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:04.706286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a88c4-829a-44de-a14b-7e37ffd757e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:04.706592 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a88c4-829a-44de-a14b-7e37ffd757e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:04.706859 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a88c4-829a-44de-a14b-7e37ffd757e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:04.716960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a88c4-829a-44de-a14b-7e37ffd757e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:04.718598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a88c4-829a-44de-a14b-7e37ffd757e7 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:04.729300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1350/2711] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:05.155597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f4f64-f0a9-44f9-a61a-064c0b0fefd6 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:05.157012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-964f4f64-f0a9-44f9-a61a-064c0b0fefd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:05.170834 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f4f64-f0a9-44f9-a61a-064c0b0fefd6 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:05.176016 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:05.176657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:05.176926 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:05.177191 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] PATH_INFO: `/v3/projects/2c85d9f01e1a49b798b8d3bd0ac573cf` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:05.178319 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c85d9f01e1a49b798b8d3bd0ac573cf)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:05.185111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:05.213948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:05.321326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c85d9f01e1a49b798b8d3bd0ac573cf', 'request_id': 'req-97b4c323-8279-4f90-9fcd-52c8f918c7e3'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.370923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c85d9f01e1a49b798b8d3bd0ac573cf', 'request_id': 'req-97b4c323-8279-4f90-9fcd-52c8f918c7e3'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.380980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c85d9f01e1a49b798b8d3bd0ac573cf', 'request_id': 'req-97b4c323-8279-4f90-9fcd-52c8f918c7e3'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.394707 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] The token cache is being invalidate because project 2c85d9f01e1a49b798b8d3bd0ac573cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:05.395383 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.395927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.398886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1362/2712] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:05 2022] DELETE /identity/v3/projects/2c85d9f01e1a49b798b8d3bd0ac573cf => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:05.403080 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a88c4-829a-44de-a14b-7e37ffd757e7 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:05.416620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a88c4-829a-44de-a14b-7e37ffd757e7 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:05.490058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:05.490772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:05.491050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:05.491442 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] PATH_INFO: `/v3/users/8fd0671bf1b24df6b15cb77336b1730e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:05.492703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd0671bf1b24df6b15cb77336b1730e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:05.494797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:05.497515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:05.690594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd0671bf1b24df6b15cb77336b1730e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.839303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd0671bf1b24df6b15cb77336b1730e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.842071 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd0671bf1b24df6b15cb77336b1730e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.853337 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd0671bf1b24df6b15cb77336b1730e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.873907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd0671bf1b24df6b15cb77336b1730e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:05.881761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1351/2713] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:05 2022] DELETE /identity/v3/users/8fd0671bf1b24df6b15cb77336b1730e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:06.624609 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:06.626093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:06.651991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4c323-8279-4f90-9fcd-52c8f918c7e3 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:06.760083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:06.760083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:06.760083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:06.760083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] PATH_INFO: `/v3/projects/5dadf6be07af45e09f5f770f2e3f7f3f` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:06.760606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dadf6be07af45e09f5f770f2e3f7f3f)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:06.760606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:06.881700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dadf6be07af45e09f5f770f2e3f7f3f', 'request_id': 'req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:06.973579 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dadf6be07af45e09f5f770f2e3f7f3f', 'request_id': 'req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:06.980998 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dadf6be07af45e09f5f770f2e3f7f3f', 'request_id': 'req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:06.996830 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] The token cache is being invalidate because project 5dadf6be07af45e09f5f770f2e3f7f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:06.997213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:06.997956 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:07.000095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1363/2714] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:06 2022] DELETE /identity/v3/projects/5dadf6be07af45e09f5f770f2e3f7f3f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:07.004470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:07.006567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:07.024171 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ad8a9-3eae-42fc-83ac-33d611a5ef7b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:07.084998 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:07.086230 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:07.086564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:07.086840 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] PATH_INFO: `/v3/users/07f7e5607f554b959c7155dee1296e1d` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:07.088266 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f7e5607f554b959c7155dee1296e1d)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:07.090081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:07.242331 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f7e5607f554b959c7155dee1296e1d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:07.323378 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f7e5607f554b959c7155dee1296e1d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:07.325569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f7e5607f554b959c7155dee1296e1d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:07.333606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f7e5607f554b959c7155dee1296e1d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:07.347567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06f3444b-02f9-43d1-a580-280fe5311f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f7e5607f554b959c7155dee1296e1d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:07.350104 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:07.354854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1352/2715] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:07 2022] DELETE /identity/v3/users/07f7e5607f554b959c7155dee1296e1d => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:08.288181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:08.290047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:08.306979 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dc22c-7810-4f12-a5c6-3d21e3e32d69 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:08.356030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:08.383533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:08.384337 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:08.389940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:08.389940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] PATH_INFO: `/v3/projects/414f8b55025e40bc9f65b0e795106ad9` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:08.389940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=414f8b55025e40bc9f65b0e795106ad9)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:08.389940 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:08.524307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414f8b55025e40bc9f65b0e795106ad9', 'request_id': 'req-b70ecc4d-dd4e-473d-a242-36b5214eeab5'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:08.579160 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414f8b55025e40bc9f65b0e795106ad9', 'request_id': 'req-b70ecc4d-dd4e-473d-a242-36b5214eeab5'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:08.586035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414f8b55025e40bc9f65b0e795106ad9', 'request_id': 'req-b70ecc4d-dd4e-473d-a242-36b5214eeab5'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:08.598381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] The token cache is being invalidate because project 414f8b55025e40bc9f65b0e795106ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:08.598672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:08.599324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b70ecc4d-dd4e-473d-a242-36b5214eeab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:08.601193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1364/2716] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:08 2022] DELETE /identity/v3/projects/414f8b55025e40bc9f65b0e795106ad9 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:08.607308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11fb46-cd34-45ec-be78-3613a33d462c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:08.607801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11fb46-cd34-45ec-be78-3613a33d462c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:08.608202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11fb46-cd34-45ec-be78-3613a33d462c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:08.629318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2f11fb46-cd34-45ec-be78-3613a33d462c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:08.633415 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2f11fb46-cd34-45ec-be78-3613a33d462c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:08.692110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1353/2717] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:08.760778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0639e3-afa6-471b-9a4f-78d4bf5da80c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:08.761237 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0639e3-afa6-471b-9a4f-78d4bf5da80c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:08.761411 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0639e3-afa6-471b-9a4f-78d4bf5da80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:08.787148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-ee0639e3-afa6-471b-9a4f-78d4bf5da80c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:08.789546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-ee0639e3-afa6-471b-9a4f-78d4bf5da80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:08.798154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1365/2718] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:08.802715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11fb46-cd34-45ec-be78-3613a33d462c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:08.814245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11fb46-cd34-45ec-be78-3613a33d462c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:08.819939 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:08.820737 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:08.820866 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:08.821081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:08.821727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:31:08.822022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:08.823713 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:08.835529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1354/2719] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:31:08.840134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0639e3-afa6-471b-9a4f-78d4bf5da80c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:08.851760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0639e3-afa6-471b-9a4f-78d4bf5da80c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:08.854457 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:08.855053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:08.855324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:08.855591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:08.856374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:08.858264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:09.012025 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1366/2720] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:31:08 2022] POST /identity/v3/projects => generated 371 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:31:09.016350 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:09.042217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2983-0b8b-4d0d-b810-badf09d8be4c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:09.134842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7e071-989d-48ac-9f8e-3c530f3db083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:09.135694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7e071-989d-48ac-9f8e-3c530f3db083 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.136133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7e071-989d-48ac-9f8e-3c530f3db083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.136563 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7e071-989d-48ac-9f8e-3c530f3db083 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.137992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7e071-989d-48ac-9f8e-3c530f3db083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:09.138143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:09.140141 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:09.140607 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7e071-989d-48ac-9f8e-3c530f3db083 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:09.254961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1355/2721] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:31:09 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:31:09.259562 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7e071-989d-48ac-9f8e-3c530f3db083 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:09.275253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7e071-989d-48ac-9f8e-3c530f3db083 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:09.285515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955996f0-a8c4-4c58-aea1-90f62ea4ab05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:09.286118 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955996f0-a8c4-4c58-aea1-90f62ea4ab05 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.286523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5477675-fc2f-40a3-8c92-c3d15bacc11f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:09.286654 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955996f0-a8c4-4c58-aea1-90f62ea4ab05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.286977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955996f0-a8c4-4c58-aea1-90f62ea4ab05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.287598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955996f0-a8c4-4c58-aea1-90f62ea4ab05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:09.290254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955996f0-a8c4-4c58-aea1-90f62ea4ab05 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:09.291349 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2785c07b-107a-4d6e-8ae9-465971059c2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:09.292030 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785c07b-107a-4d6e-8ae9-465971059c2d service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.292315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785c07b-107a-4d6e-8ae9-465971059c2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.292594 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785c07b-107a-4d6e-8ae9-465971059c2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.294629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2785c07b-107a-4d6e-8ae9-465971059c2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:09.307801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1356/2722] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:31:09.309589 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955996f0-a8c4-4c58-aea1-90f62ea4ab05 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:09.322133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955996f0-a8c4-4c58-aea1-90f62ea4ab05 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:09.328202 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c70443-5f84-4396-b027-6d2cd1bef376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:09.328861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c70443-5f84-4396-b027-6d2cd1bef376 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.329125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c70443-5f84-4396-b027-6d2cd1bef376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.329402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c70443-5f84-4396-b027-6d2cd1bef376 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.330051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c70443-5f84-4396-b027-6d2cd1bef376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:09.331691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c70443-5f84-4396-b027-6d2cd1bef376 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:09.347219 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1357/2723] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:31:09.348507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c70443-5f84-4396-b027-6d2cd1bef376 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:09.353615 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:09.360738 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c70443-5f84-4396-b027-6d2cd1bef376 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:09.366011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c0a32-ac0f-4f74-8778-738a1de3c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:09.367583 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0a32-ac0f-4f74-8778-738a1de3c767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.368010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0a32-ac0f-4f74-8778-738a1de3c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.368292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c0a32-ac0f-4f74-8778-738a1de3c767 admin admin] PATH_INFO: `/v3/projects/443311f53bdd45c6b79007120dcef73f/users/0e119f7cf3f245f9812d04a2743b9428/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.385284 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c0a32-ac0f-4f74-8778-738a1de3c767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=443311f53bdd45c6b79007120dcef73f, user_id=0e119f7cf3f245f9812d04a2743b9428, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:09.388181 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c0a32-ac0f-4f74-8778-738a1de3c767 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:09.412539 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785c07b-107a-4d6e-8ae9-465971059c2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:09.415181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2785c07b-107a-4d6e-8ae9-465971059c2d service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:09.452086 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1367/2724] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:31:09.459186 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1358/2725] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:31:09 2022] PUT /identity/v3/projects/443311f53bdd45c6b79007120dcef73f/users/0e119f7cf3f245f9812d04a2743b9428/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:31:09.470464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657de98-ef47-4f46-99eb-fb04605752db None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.470810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657de98-ef47-4f46-99eb-fb04605752db None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.471428 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657de98-ef47-4f46-99eb-fb04605752db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.520219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-9657de98-ef47-4f46-99eb-fb04605752db None None] MFA Rules not processed for user `0e119f7cf3f245f9812d04a2743b9428`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:09.523475 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9657de98-ef47-4f46-99eb-fb04605752db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:09.629257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1368/2726] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:09.637135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c0a32-ac0f-4f74-8778-738a1de3c767 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:09.689476 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:09.711559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c0a32-ac0f-4f74-8778-738a1de3c767 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:09.715172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:09.715763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.716038 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.716303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.724318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:09.726246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:09.734850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1359/2727] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:09.757977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657de98-ef47-4f46-99eb-fb04605752db None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:09.766999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:09.768588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:09.842198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657de98-ef47-4f46-99eb-fb04605752db None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:09.847424 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:09.848049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.848321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.848587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] PATH_INFO: `/v3/users/0e119f7cf3f245f9812d04a2743b9428` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.849722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e119f7cf3f245f9812d04a2743b9428)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:09.852722 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:09.861390 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fccfb-8c1a-448d-bd6a-9d1a18b22ad9 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:09.866431 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:09.867302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:09.867579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:09.867851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] PATH_INFO: `/v3/users/cf9d4a796a8c450486832bb10c3d4ad0` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:09.868988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9d4a796a8c450486832bb10c3d4ad0)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:09.870925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:10.036253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:10.044804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e119f7cf3f245f9812d04a2743b9428'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.045440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e119f7cf3f245f9812d04a2743b9428'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.070869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9d4a796a8c450486832bb10c3d4ad0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.144970 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e119f7cf3f245f9812d04a2743b9428'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.157823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e119f7cf3f245f9812d04a2743b9428'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.165027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-40da4924-2a5b-4c49-882c-02664ea4cce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e119f7cf3f245f9812d04a2743b9428'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.171434 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1369/2728] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:09 2022] DELETE /identity/v3/users/0e119f7cf3f245f9812d04a2743b9428 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:10.175662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a669a-545e-4f2a-96dd-72ddb2b6e250 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:10.175946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a669a-545e-4f2a-96dd-72ddb2b6e250 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:10.176221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a669a-545e-4f2a-96dd-72ddb2b6e250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:10.196172 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a44a669a-545e-4f2a-96dd-72ddb2b6e250 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:10.197400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a44a669a-545e-4f2a-96dd-72ddb2b6e250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:10.219298 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9d4a796a8c450486832bb10c3d4ad0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.219972 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9d4a796a8c450486832bb10c3d4ad0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.228156 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9d4a796a8c450486832bb10c3d4ad0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.240982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9d4a796a8c450486832bb10c3d4ad0'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.247599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1360/2729] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:09 2022] DELETE /identity/v3/users/cf9d4a796a8c450486832bb10c3d4ad0 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:31:10.256117 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1370/2730] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:31:10.266735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:10.268204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:10.328221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb5e7e-014c-42a8-beb2-2393f7240bc1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:10.332665 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ae268-0e1f-402c-858f-a750d99cea3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:10.333275 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ae268-0e1f-402c-858f-a750d99cea3f service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:10.333731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ae268-0e1f-402c-858f-a750d99cea3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:10.334004 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ae268-0e1f-402c-858f-a750d99cea3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:10.342039 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:10.402160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ae268-0e1f-402c-858f-a750d99cea3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:10.404443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ae268-0e1f-402c-858f-a750d99cea3f service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:10.422407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1361/2731] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:10.550443 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a669a-545e-4f2a-96dd-72ddb2b6e250 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:10.550904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a44a669a-545e-4f2a-96dd-72ddb2b6e250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:10.565995 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a669a-545e-4f2a-96dd-72ddb2b6e250 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:10.571616 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:10.572448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:10.572724 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:10.573015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] PATH_INFO: `/v3/projects/602875e689b7476697ce2e9bbc3bf7b5` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:10.574236 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=602875e689b7476697ce2e9bbc3bf7b5)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:10.576045 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:10.748862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '602875e689b7476697ce2e9bbc3bf7b5', 'request_id': 'req-e9365c92-5f19-427e-b11d-08fd6b1231c2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.815678 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '602875e689b7476697ce2e9bbc3bf7b5', 'request_id': 'req-e9365c92-5f19-427e-b11d-08fd6b1231c2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.822476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '602875e689b7476697ce2e9bbc3bf7b5', 'request_id': 'req-e9365c92-5f19-427e-b11d-08fd6b1231c2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.833896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] The token cache is being invalidate because project 602875e689b7476697ce2e9bbc3bf7b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:10.834178 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.834863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:10.836534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1371/2732] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:10 2022] DELETE /identity/v3/projects/602875e689b7476697ce2e9bbc3bf7b5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:10.840656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ae268-0e1f-402c-858f-a750d99cea3f service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:10.851210 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ae268-0e1f-402c-858f-a750d99cea3f service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:10.856229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:10.860860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:10.910590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:10.940322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:10.941158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:10.941436 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:10.941705 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] PATH_INFO: `/v3/users/9c26d229acd5487e807bd74eaaaba535` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:10.942968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c26d229acd5487e807bd74eaaaba535)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:10.945116 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:10.950021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9365c92-5f19-427e-b11d-08fd6b1231c2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:10.955732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:10.956440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:10.956946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:10.957226 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] PATH_INFO: `/v3/projects/443311f53bdd45c6b79007120dcef73f` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:10.958401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=443311f53bdd45c6b79007120dcef73f)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:10.961235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:11.244420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c26d229acd5487e807bd74eaaaba535'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.244625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '443311f53bdd45c6b79007120dcef73f', 'request_id': 'req-f9b7ceee-b882-4b73-b23e-48cca44da55b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.318826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '443311f53bdd45c6b79007120dcef73f', 'request_id': 'req-f9b7ceee-b882-4b73-b23e-48cca44da55b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.325397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '443311f53bdd45c6b79007120dcef73f', 'request_id': 'req-f9b7ceee-b882-4b73-b23e-48cca44da55b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.343078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] The token cache is being invalidate because project 443311f53bdd45c6b79007120dcef73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:11.343517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.344329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-f9b7ceee-b882-4b73-b23e-48cca44da55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.346127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1372/2733] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:10 2022] DELETE /identity/v3/projects/443311f53bdd45c6b79007120dcef73f => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:11.351350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e044110-6099-4546-b090-ee4ba9eac076 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:11.355394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e044110-6099-4546-b090-ee4ba9eac076 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:11.355394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e044110-6099-4546-b090-ee4ba9eac076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:11.371805 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-8e044110-6099-4546-b090-ee4ba9eac076 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:11.373009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8e044110-6099-4546-b090-ee4ba9eac076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:11.377144 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c26d229acd5487e807bd74eaaaba535'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.377832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c26d229acd5487e807bd74eaaaba535'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.386392 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c26d229acd5487e807bd74eaaaba535'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.399420 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-d349a2ca-15a2-4326-b507-a5ac75c1cb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c26d229acd5487e807bd74eaaaba535'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.406698 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1362/2734] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:10 2022] DELETE /identity/v3/users/9c26d229acd5487e807bd74eaaaba535 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:11.451900 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1373/2735] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:31:11.459770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490ea2-8f0c-45b1-b39a-a5033360fd25 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:11.460996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490ea2-8f0c-45b1-b39a-a5033360fd25 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:11.461429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5490ea2-8f0c-45b1-b39a-a5033360fd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:11.483682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-d5490ea2-8f0c-45b1-b39a-a5033360fd25 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:11.485054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d5490ea2-8f0c-45b1-b39a-a5033360fd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:11.505761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:11.557143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1363/2736] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:11.561649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e044110-6099-4546-b090-ee4ba9eac076 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:11.574071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e044110-6099-4546-b090-ee4ba9eac076 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:11.578911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:11.579511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:11.579831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:11.580239 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:11.580914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:31:11.581277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:11.582939 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:11.595534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1374/2737] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:31:11.599934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5490ea2-8f0c-45b1-b39a-a5033360fd25 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:11.609636 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5490ea2-8f0c-45b1-b39a-a5033360fd25 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:11.614775 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc82b05-e3d0-472c-bf72-bf8cf7624a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:11.615374 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82b05-e3d0-472c-bf72-bf8cf7624a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:11.615650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82b05-e3d0-472c-bf72-bf8cf7624a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:11.615920 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc82b05-e3d0-472c-bf72-bf8cf7624a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:11.617279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc82b05-e3d0-472c-bf72-bf8cf7624a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:11.619353 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc82b05-e3d0-472c-bf72-bf8cf7624a03 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:11.701524 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:11.703368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:11.718005 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d1104-c46d-4d5d-a931-cdf7ed7cfb5d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:11.723012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:11.723645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:11.723917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:11.724184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] PATH_INFO: `/v3/projects/87c007e6d40d40e3b268dfd2fe3a0d7d` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:11.725320 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c007e6d40d40e3b268dfd2fe3a0d7d)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:11.727658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:11.735326 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1364/2738] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:31:11 2022] POST /identity/v3/projects => generated 359 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:31:11.739203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc82b05-e3d0-472c-bf72-bf8cf7624a03 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:11.801983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc82b05-e3d0-472c-bf72-bf8cf7624a03 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:11.807366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88a2dc-b323-4e22-87d6-292e4b343d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:11.807968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88a2dc-b323-4e22-87d6-292e4b343d2b admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:11.808246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88a2dc-b323-4e22-87d6-292e4b343d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:11.808514 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88a2dc-b323-4e22-87d6-292e4b343d2b admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:11.809130 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88a2dc-b323-4e22-87d6-292e4b343d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:11.811108 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88a2dc-b323-4e22-87d6-292e4b343d2b admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:11.839168 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c007e6d40d40e3b268dfd2fe3a0d7d', 'request_id': 'req-0801c709-72dd-45fa-b502-15d5a05e229a'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.840799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:11.939256 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1365/2739] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:31:11 2022] POST /identity/v3/users => generated 280 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:31:11.943332 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c007e6d40d40e3b268dfd2fe3a0d7d', 'request_id': 'req-0801c709-72dd-45fa-b502-15d5a05e229a'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.943837 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88a2dc-b323-4e22-87d6-292e4b343d2b admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:11.950465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c007e6d40d40e3b268dfd2fe3a0d7d', 'request_id': 'req-0801c709-72dd-45fa-b502-15d5a05e229a'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.954136 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88a2dc-b323-4e22-87d6-292e4b343d2b admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:11.962496 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] The token cache is being invalidate because project 87c007e6d40d40e3b268dfd2fe3a0d7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:11.962778 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.965747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:11.965747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1375/2740] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:11 2022] DELETE /identity/v3/projects/87c007e6d40d40e3b268dfd2fe3a0d7d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:31:12.026228 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82712b26-af96-462d-82fb-1425836b2a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:12.026870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82712b26-af96-462d-82fb-1425836b2a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:12.027143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82712b26-af96-462d-82fb-1425836b2a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:12.027437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82712b26-af96-462d-82fb-1425836b2a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:12.028074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82712b26-af96-462d-82fb-1425836b2a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:12.029745 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82712b26-af96-462d-82fb-1425836b2a63 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:12.041571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1366/2741] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:31:12.046275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:12.047470 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:12.064053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801c709-72dd-45fa-b502-15d5a05e229a admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:12.068898 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b11a9-e96b-41ba-9daa-e56ed3cf5a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:12.069965 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b11a9-e96b-41ba-9daa-e56ed3cf5a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:12.070309 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b11a9-e96b-41ba-9daa-e56ed3cf5a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:12.070692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b11a9-e96b-41ba-9daa-e56ed3cf5a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:12.071415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b11a9-e96b-41ba-9daa-e56ed3cf5a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:12.073120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b11a9-e96b-41ba-9daa-e56ed3cf5a28 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:12.084698 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1376/2742] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:31:12.089668 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82712b26-af96-462d-82fb-1425836b2a63 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:12.100220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82712b26-af96-462d-82fb-1425836b2a63 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:12.105439 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:12.106100 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:12.106401 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:12.106671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] PATH_INFO: `/v3/projects/301d35da7d8e4db696dcf919a69a994c/users/e738707ed1d7452cab25e81aabdd0717/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:12.123207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301d35da7d8e4db696dcf919a69a994c, user_id=e738707ed1d7452cab25e81aabdd0717, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:12.125876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:12.200204 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1367/2743] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:31:12 2022] PUT /identity/v3/projects/301d35da7d8e4db696dcf919a69a994c/users/e738707ed1d7452cab25e81aabdd0717/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:31:12.213265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149040cd-c12e-4abd-bfa9-61b003df321a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:12.213715 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149040cd-c12e-4abd-bfa9-61b003df321a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:12.213989 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149040cd-c12e-4abd-bfa9-61b003df321a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:12.242841 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-149040cd-c12e-4abd-bfa9-61b003df321a None None] MFA Rules not processed for user `e738707ed1d7452cab25e81aabdd0717`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:12.244039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-149040cd-c12e-4abd-bfa9-61b003df321a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:12.306427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1377/2744] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:12.341475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:12.343194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:12.348412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:12.400756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e79be5-5366-43d6-bd08-a9c3517086c2 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:12.404400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246c289-56a3-417c-8d7e-a03d3991a50a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:12.404992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c289-56a3-417c-8d7e-a03d3991a50a service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:12.405270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c289-56a3-417c-8d7e-a03d3991a50a service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:12.405534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246c289-56a3-417c-8d7e-a03d3991a50a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:12.413861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246c289-56a3-417c-8d7e-a03d3991a50a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:12.415343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246c289-56a3-417c-8d7e-a03d3991a50a service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:12.424888 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1368/2745] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:12.440235 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149040cd-c12e-4abd-bfa9-61b003df321a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:12.502162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149040cd-c12e-4abd-bfa9-61b003df321a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:12.507377 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:12.507994 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:12.508295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:12.508528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] PATH_INFO: `/v3/users/e738707ed1d7452cab25e81aabdd0717` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:12.509661 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e738707ed1d7452cab25e81aabdd0717)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:12.511614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:12.651023 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e738707ed1d7452cab25e81aabdd0717'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:12.652533 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e738707ed1d7452cab25e81aabdd0717'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:12.699215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e738707ed1d7452cab25e81aabdd0717'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:12.704144 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:12.712588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e738707ed1d7452cab25e81aabdd0717'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:12.719888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e738707ed1d7452cab25e81aabdd0717'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:12.726740 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1378/2746] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:12 2022] DELETE /identity/v3/users/e738707ed1d7452cab25e81aabdd0717 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:12.731470 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84770d-329a-4785-91ee-17e5dc89aa44 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:12.731783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84770d-329a-4785-91ee-17e5dc89aa44 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:12.732056 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84770d-329a-4785-91ee-17e5dc89aa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:12.752766 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-0c84770d-329a-4785-91ee-17e5dc89aa44 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:12.754023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0c84770d-329a-4785-91ee-17e5dc89aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:12.815550 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1369/2747] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:31:12.825505 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:12.827060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:12.883280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b5ca92-0a58-472d-89ab-0c44291765d2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:12.886893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb5a14-cdde-4594-bb52-9cf0ac984afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:12.887493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb5a14-cdde-4594-bb52-9cf0ac984afe service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:12.887769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb5a14-cdde-4594-bb52-9cf0ac984afe service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:12.888039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb5a14-cdde-4594-bb52-9cf0ac984afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:12.897169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb5a14-cdde-4594-bb52-9cf0ac984afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:12.898745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb5a14-cdde-4594-bb52-9cf0ac984afe service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:12.908094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1379/2748] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:13.338013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84770d-329a-4785-91ee-17e5dc89aa44 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:13.339691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-0c84770d-329a-4785-91ee-17e5dc89aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:13.363534 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84770d-329a-4785-91ee-17e5dc89aa44 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:13.369464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:13.370576 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:13.370962 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:13.371397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] PATH_INFO: `/v3/projects/301d35da7d8e4db696dcf919a69a994c` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:13.372957 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=301d35da7d8e4db696dcf919a69a994c)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:13.375010 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:13.501348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301d35da7d8e4db696dcf919a69a994c', 'request_id': 'req-42665db7-4b58-4a85-aacb-0111f8f0a8ab'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:13.550294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301d35da7d8e4db696dcf919a69a994c', 'request_id': 'req-42665db7-4b58-4a85-aacb-0111f8f0a8ab'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:13.558651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301d35da7d8e4db696dcf919a69a994c', 'request_id': 'req-42665db7-4b58-4a85-aacb-0111f8f0a8ab'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:13.571943 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] The token cache is being invalidate because project 301d35da7d8e4db696dcf919a69a994c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:13.572277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:13.572990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-42665db7-4b58-4a85-aacb-0111f8f0a8ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:13.574629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1370/2749] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:13 2022] DELETE /identity/v3/projects/301d35da7d8e4db696dcf919a69a994c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:13.579406 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fbcb3-1d92-4b71-bd77-d204127f7f32 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:13.579746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fbcb3-1d92-4b71-bd77-d204127f7f32 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:13.580054 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fbcb3-1d92-4b71-bd77-d204127f7f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:13.602277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-386fbcb3-1d92-4b71-bd77-d204127f7f32 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:13.603586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-386fbcb3-1d92-4b71-bd77-d204127f7f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:13.664915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1380/2750] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:13.672352 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2285a-09b0-4c23-9c7c-a65c1021a550 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:13.673166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2285a-09b0-4c23-9c7c-a65c1021a550 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:13.673166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2285a-09b0-4c23-9c7c-a65c1021a550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:13.693129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f1a2285a-09b0-4c23-9c7c-a65c1021a550 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:13.695074 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f1a2285a-09b0-4c23-9c7c-a65c1021a550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:13.703501 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1371/2751] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:13.708651 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fbcb3-1d92-4b71-bd77-d204127f7f32 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:13.718675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fbcb3-1d92-4b71-bd77-d204127f7f32 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:13.723400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:13.723996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:13.724273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:13.724548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:13.725218 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:31:13.726051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:13.727673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:13.739301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1381/2752] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:31:13.744264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2285a-09b0-4c23-9c7c-a65c1021a550 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:13.753629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2285a-09b0-4c23-9c7c-a65c1021a550 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:13.758521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2701c7-aac3-4cb9-b92f-d3637e8bfe8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:13.765729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2701c7-aac3-4cb9-b92f-d3637e8bfe8e admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:13.765729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2701c7-aac3-4cb9-b92f-d3637e8bfe8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:13.765729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2701c7-aac3-4cb9-b92f-d3637e8bfe8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:13.765729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2701c7-aac3-4cb9-b92f-d3637e8bfe8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:13.765729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2701c7-aac3-4cb9-b92f-d3637e8bfe8e admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:13.849055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:13.857818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1372/2753] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:31:13 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:31:13.862264 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:13.872514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60073cea-f47f-490b-8be6-9622c02f4d5b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:13.933981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703ad5f-3e1a-487a-a30e-41b81af66124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:13.934567 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703ad5f-3e1a-487a-a30e-41b81af66124 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:13.934847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703ad5f-3e1a-487a-a30e-41b81af66124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:13.935183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703ad5f-3e1a-487a-a30e-41b81af66124 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:13.935816 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703ad5f-3e1a-487a-a30e-41b81af66124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:13.937748 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703ad5f-3e1a-487a-a30e-41b81af66124 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:13.948997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:14.010219 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1382/2754] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:31:13 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:31:14.014412 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2701c7-aac3-4cb9-b92f-d3637e8bfe8e admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:14.025489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2701c7-aac3-4cb9-b92f-d3637e8bfe8e admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:14.030323 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c74d8-282d-45b9-8627-cd59abf38d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:14.030926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c74d8-282d-45b9-8627-cd59abf38d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.031199 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c74d8-282d-45b9-8627-cd59abf38d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.031464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c74d8-282d-45b9-8627-cd59abf38d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.032086 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c74d8-282d-45b9-8627-cd59abf38d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:14.033671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c74d8-282d-45b9-8627-cd59abf38d59 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:14.044606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1373/2755] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:31:14.048620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703ad5f-3e1a-487a-a30e-41b81af66124 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:14.059097 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703ad5f-3e1a-487a-a30e-41b81af66124 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:14.064211 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a1794-4c4d-450b-9dcf-31aa0ad87127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:14.064812 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a1794-4c4d-450b-9dcf-31aa0ad87127 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.065181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a1794-4c4d-450b-9dcf-31aa0ad87127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.065448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a1794-4c4d-450b-9dcf-31aa0ad87127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.066078 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a1794-4c4d-450b-9dcf-31aa0ad87127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:14.067785 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a1794-4c4d-450b-9dcf-31aa0ad87127 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:14.079886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1383/2756] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:31:14.088757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c74d8-282d-45b9-8627-cd59abf38d59 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:14.099641 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c74d8-282d-45b9-8627-cd59abf38d59 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:14.107541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfd559-aa5d-4953-a3bf-0209b0526003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:14.108217 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd559-aa5d-4953-a3bf-0209b0526003 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.108492 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd559-aa5d-4953-a3bf-0209b0526003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.108762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd559-aa5d-4953-a3bf-0209b0526003 admin admin] PATH_INFO: `/v3/projects/0742a538877c4743993c21d145c62ee1/users/6d034a1c850645a69a61fcd393d84340/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.126716 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfd559-aa5d-4953-a3bf-0209b0526003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0742a538877c4743993c21d145c62ee1, user_id=6d034a1c850645a69a61fcd393d84340, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:14.129586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfd559-aa5d-4953-a3bf-0209b0526003 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:14.249231 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1374/2757] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:31:14 2022] PUT /identity/v3/projects/0742a538877c4743993c21d145c62ee1/users/6d034a1c850645a69a61fcd393d84340/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:14.260910 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8104c6-8f46-4692-bb05-317f4330e1fa None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.261194 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8104c6-8f46-4692-bb05-317f4330e1fa None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.261566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8104c6-8f46-4692-bb05-317f4330e1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.293325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-3c8104c6-8f46-4692-bb05-317f4330e1fa None None] MFA Rules not processed for user `6d034a1c850645a69a61fcd393d84340`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:14.297817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3c8104c6-8f46-4692-bb05-317f4330e1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:14.375566 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1384/2758] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:14.380313 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfd559-aa5d-4953-a3bf-0209b0526003 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:14.455854 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfd559-aa5d-4953-a3bf-0209b0526003 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:14.460688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006337d-9d2d-4451-81a1-221b7e197f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:14.461292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006337d-9d2d-4451-81a1-221b7e197f5f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.461569 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006337d-9d2d-4451-81a1-221b7e197f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.461841 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006337d-9d2d-4451-81a1-221b7e197f5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.462513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006337d-9d2d-4451-81a1-221b7e197f5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:14.464467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006337d-9d2d-4451-81a1-221b7e197f5f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:14.583185 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1375/2759] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:31:14 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:31:14.588871 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8104c6-8f46-4692-bb05-317f4330e1fa None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:14.599860 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8104c6-8f46-4692-bb05-317f4330e1fa None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:14.664427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deafa85-6f02-4e98-9f3d-6ce533493d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:14.665043 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deafa85-6f02-4e98-9f3d-6ce533493d49 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.665580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deafa85-6f02-4e98-9f3d-6ce533493d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.665580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deafa85-6f02-4e98-9f3d-6ce533493d49 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.666181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deafa85-6f02-4e98-9f3d-6ce533493d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:14.668234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deafa85-6f02-4e98-9f3d-6ce533493d49 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:14.766456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1385/2760] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:31:14 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:31:14.770440 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006337d-9d2d-4451-81a1-221b7e197f5f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:14.781126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006337d-9d2d-4451-81a1-221b7e197f5f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:14.786190 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684405b-69e8-444e-bb77-c8edcda37c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:14.786808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684405b-69e8-444e-bb77-c8edcda37c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.787182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684405b-69e8-444e-bb77-c8edcda37c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.787479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684405b-69e8-444e-bb77-c8edcda37c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.788629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684405b-69e8-444e-bb77-c8edcda37c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:14.790984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684405b-69e8-444e-bb77-c8edcda37c25 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:14.802237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1376/2761] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:31:14.807351 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deafa85-6f02-4e98-9f3d-6ce533493d49 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:14.817460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deafa85-6f02-4e98-9f3d-6ce533493d49 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:14.823282 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f41b40-1d59-46d3-93bc-7691decdb764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:14.825801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41b40-1d59-46d3-93bc-7691decdb764 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.825801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41b40-1d59-46d3-93bc-7691decdb764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.825801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41b40-1d59-46d3-93bc-7691decdb764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.825801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f41b40-1d59-46d3-93bc-7691decdb764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:14.826669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f41b40-1d59-46d3-93bc-7691decdb764 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:14.836881 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1386/2762] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:31:14.841216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684405b-69e8-444e-bb77-c8edcda37c25 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:14.851608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684405b-69e8-444e-bb77-c8edcda37c25 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:14.856995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93f20c-0e79-49ef-9936-e3477af0ff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:14.856995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93f20c-0e79-49ef-9936-e3477af0ff50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.857423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93f20c-0e79-49ef-9936-e3477af0ff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.857531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93f20c-0e79-49ef-9936-e3477af0ff50 admin admin] PATH_INFO: `/v3/projects/9cb828d5a46d42858cf80fe6ea329f06/users/02bdc5bdd939453f8ca76c56282f302e/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:14.874292 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93f20c-0e79-49ef-9936-e3477af0ff50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb828d5a46d42858cf80fe6ea329f06, user_id=02bdc5bdd939453f8ca76c56282f302e, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:14.877001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93f20c-0e79-49ef-9936-e3477af0ff50 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:14.965893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1377/2763] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:31:14 2022] PUT /identity/v3/projects/9cb828d5a46d42858cf80fe6ea329f06/users/02bdc5bdd939453f8ca76c56282f302e/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:31:14.978056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba459-b74e-4656-b29e-0e493ba3a8e2 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:14.978447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba459-b74e-4656-b29e-0e493ba3a8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:14.978723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dba459-b74e-4656-b29e-0e493ba3a8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.012432 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-f0dba459-b74e-4656-b29e-0e493ba3a8e2 None None] MFA Rules not processed for user `02bdc5bdd939453f8ca76c56282f302e`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:15.013821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-f0dba459-b74e-4656-b29e-0e493ba3a8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:15.071338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:15.106543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1387/2764] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:15.111214 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93f20c-0e79-49ef-9936-e3477af0ff50 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:15.179645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93f20c-0e79-49ef-9936-e3477af0ff50 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:15.184291 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5768ed-d742-4bea-8867-f2ac0e0a8170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:15.184889 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5768ed-d742-4bea-8867-f2ac0e0a8170 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:15.187237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5768ed-d742-4bea-8867-f2ac0e0a8170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:15.187237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5768ed-d742-4bea-8867-f2ac0e0a8170 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.187237 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5768ed-d742-4bea-8867-f2ac0e0a8170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:15.187934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5768ed-d742-4bea-8867-f2ac0e0a8170 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:15.283286 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:15.290407 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1378/2765] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:31:15 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:31:15.294658 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dba459-b74e-4656-b29e-0e493ba3a8e2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:15.305689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dba459-b74e-4656-b29e-0e493ba3a8e2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:15.364152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c1ddf-e6ed-418a-b82e-9f5b80f39a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:15.364742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1ddf-e6ed-418a-b82e-9f5b80f39a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:15.365014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1ddf-e6ed-418a-b82e-9f5b80f39a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:15.365284 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c1ddf-e6ed-418a-b82e-9f5b80f39a1d admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.365917 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c1ddf-e6ed-418a-b82e-9f5b80f39a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:15.368363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c1ddf-e6ed-418a-b82e-9f5b80f39a1d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:15.466089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1388/2766] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:31:15 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:31:15.470103 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5768ed-d742-4bea-8867-f2ac0e0a8170 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:15.481309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5768ed-d742-4bea-8867-f2ac0e0a8170 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:15.485909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9061e-853e-4e32-82bb-af33bed03067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:15.486598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9061e-853e-4e32-82bb-af33bed03067 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:15.492362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9061e-853e-4e32-82bb-af33bed03067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:15.492362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9061e-853e-4e32-82bb-af33bed03067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.492362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9061e-853e-4e32-82bb-af33bed03067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:15.492362 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9061e-853e-4e32-82bb-af33bed03067 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:15.500532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1379/2767] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:31:15.504905 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c1ddf-e6ed-418a-b82e-9f5b80f39a1d admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:15.515170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c1ddf-e6ed-418a-b82e-9f5b80f39a1d admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:15.519846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4c4dc-fe2a-415a-aa96-b5bf03c548eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:15.520493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4c4dc-fe2a-415a-aa96-b5bf03c548eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:15.520769 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4c4dc-fe2a-415a-aa96-b5bf03c548eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:15.521040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf4c4dc-fe2a-415a-aa96-b5bf03c548eb admin admin] PATH_INFO: `/v3/projects/307b98abe5be4b789570a60e3d0d327a/users/9b11bf7b044c498caa5f30a22513ac8a/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.536521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4c4dc-fe2a-415a-aa96-b5bf03c548eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=307b98abe5be4b789570a60e3d0d327a, user_id=9b11bf7b044c498caa5f30a22513ac8a, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:15.539862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf4c4dc-fe2a-415a-aa96-b5bf03c548eb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:15.620415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1389/2768] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:31:15 2022] PUT /identity/v3/projects/307b98abe5be4b789570a60e3d0d327a/users/9b11bf7b044c498caa5f30a22513ac8a/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:15.624172 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9061e-853e-4e32-82bb-af33bed03067 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:15.634666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9061e-853e-4e32-82bb-af33bed03067 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:15.695725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a169f-0c96-4147-b714-43a7c6be5244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:15.696366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a169f-0c96-4147-b714-43a7c6be5244 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:15.696645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a169f-0c96-4147-b714-43a7c6be5244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:15.696919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a169f-0c96-4147-b714-43a7c6be5244 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.697586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a169f-0c96-4147-b714-43a7c6be5244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:15.699225 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a169f-0c96-4147-b714-43a7c6be5244 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:15.709612 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1380/2769] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:31:15.714861 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4c4dc-fe2a-415a-aa96-b5bf03c548eb admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:15.725169 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf4c4dc-fe2a-415a-aa96-b5bf03c548eb admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:15.730865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:15.731531 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:15.731807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:15.732077 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] PATH_INFO: `/v3/projects/307b98abe5be4b789570a60e3d0d327a/users/9b11bf7b044c498caa5f30a22513ac8a/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.734003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=307b98abe5be4b789570a60e3d0d327a, user_id=9b11bf7b044c498caa5f30a22513ac8a, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:15.736675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:15.747587 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1390/2770] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:31:15 2022] PUT /identity/v3/projects/307b98abe5be4b789570a60e3d0d327a/users/9b11bf7b044c498caa5f30a22513ac8a/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:15.758169 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285ec25-2acd-4109-bffe-654e0f62a023 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:15.758507 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285ec25-2acd-4109-bffe-654e0f62a023 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:15.758784 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285ec25-2acd-4109-bffe-654e0f62a023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.787285 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2285ec25-2acd-4109-bffe-654e0f62a023 None None] MFA Rules not processed for user `9b11bf7b044c498caa5f30a22513ac8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:15.788517 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2285ec25-2acd-4109-bffe-654e0f62a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:15.855234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1381/2771] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:15.873101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:15.874781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:15.931810 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022936a9-095e-47d5-9853-529cfa150946 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:15.935418 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bf474-599b-4d1c-83eb-88f74f54e564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:15.936004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bf474-599b-4d1c-83eb-88f74f54e564 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:15.936277 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bf474-599b-4d1c-83eb-88f74f54e564 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:15.936540 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bf474-599b-4d1c-83eb-88f74f54e564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:15.982580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:15.992090 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bf474-599b-4d1c-83eb-88f74f54e564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:15.993498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bf474-599b-4d1c-83eb-88f74f54e564 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:16.002065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1391/2772] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:17.909846 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285ec25-2acd-4109-bffe-654e0f62a023 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:17.911329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2285ec25-2acd-4109-bffe-654e0f62a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:17.924154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285ec25-2acd-4109-bffe-654e0f62a023 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:17.928145 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ddf47-f9d9-471b-bdb7-9323a92e0ac3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:17.928741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ddf47-f9d9-471b-bdb7-9323a92e0ac3 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:17.929014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ddf47-f9d9-471b-bdb7-9323a92e0ac3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:17.929278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ddf47-f9d9-471b-bdb7-9323a92e0ac3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:17.988475 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ddf47-f9d9-471b-bdb7-9323a92e0ac3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:17.989904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ddf47-f9d9-471b-bdb7-9323a92e0ac3 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:17.999203 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1382/2773] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:19.402454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f900bd-867b-4955-9382-41006ac1155e None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:19.402996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f900bd-867b-4955-9382-41006ac1155e None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:19.402996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f900bd-867b-4955-9382-41006ac1155e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:19.422279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-a4f900bd-867b-4955-9382-41006ac1155e None None] MFA Rules not processed for user `592da799f0c540d68a2cc6d54006467c`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:19.423663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a4f900bd-867b-4955-9382-41006ac1155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:19.482175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1392/2774] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:19.608684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ddf47-f9d9-471b-bdb7-9323a92e0ac3 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:19.618396 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ddf47-f9d9-471b-bdb7-9323a92e0ac3 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:19.621987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2d233-9c01-4e84-88d0-85c5f1c1ad27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:19.622609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2d233-9c01-4e84-88d0-85c5f1c1ad27 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:19.622883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2d233-9c01-4e84-88d0-85c5f1c1ad27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:19.623148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2d233-9c01-4e84-88d0-85c5f1c1ad27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:19.630067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:19.632666 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2d233-9c01-4e84-88d0-85c5f1c1ad27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:19.634110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2d233-9c01-4e84-88d0-85c5f1c1ad27 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:19.644259 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1383/2775] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:20.231116 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f900bd-867b-4955-9382-41006ac1155e None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:20.232606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a4f900bd-867b-4955-9382-41006ac1155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:20.294731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f900bd-867b-4955-9382-41006ac1155e None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:20.300467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314e548-69e0-4e6d-8d67-22320a0392fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:20.300467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e548-69e0-4e6d-8d67-22320a0392fb service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:20.300467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e548-69e0-4e6d-8d67-22320a0392fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:20.300467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e548-69e0-4e6d-8d67-22320a0392fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:20.301588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8314e548-69e0-4e6d-8d67-22320a0392fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:20.377835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314e548-69e0-4e6d-8d67-22320a0392fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:20.379328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314e548-69e0-4e6d-8d67-22320a0392fb service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:20.391079 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1393/2776] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:24.365444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2d233-9c01-4e84-88d0-85c5f1c1ad27 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:24.430429 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2d233-9c01-4e84-88d0-85c5f1c1ad27 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:24.435894 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:24.436529 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:24.436804 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:24.437073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] PATH_INFO: `/v3/users/6d034a1c850645a69a61fcd393d84340` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:24.438193 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d034a1c850645a69a61fcd393d84340)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:24.440148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:24.618917 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d034a1c850645a69a61fcd393d84340'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:24.701755 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d034a1c850645a69a61fcd393d84340'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:24.702423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d034a1c850645a69a61fcd393d84340'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:24.710387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d034a1c850645a69a61fcd393d84340'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:24.723389 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d034a1c850645a69a61fcd393d84340'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:24.732143 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1384/2777] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:24 2022] DELETE /identity/v3/users/6d034a1c850645a69a61fcd393d84340 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:24.736608 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c8d4-492e-449a-b0da-7ccb33d1c9d2 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:24.737080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c8d4-492e-449a-b0da-7ccb33d1c9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:24.737441 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2c8d4-492e-449a-b0da-7ccb33d1c9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:24.757400 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-75b2c8d4-492e-449a-b0da-7ccb33d1c9d2 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:24.758689 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-75b2c8d4-492e-449a-b0da-7ccb33d1c9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:24.789096 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=100 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:24.819159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1394/2778] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:24.829051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:24.830564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:24.890226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006212f8-c377-4978-9add-93f2976cacff admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:24.893851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9ff7-93e0-495e-9d76-f8bce2783770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:24.894519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9ff7-93e0-495e-9d76-f8bce2783770 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:24.894795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9ff7-93e0-495e-9d76-f8bce2783770 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:24.895057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d9ff7-93e0-495e-9d76-f8bce2783770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:24.905279 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d9ff7-93e0-495e-9d76-f8bce2783770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:24.906772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d9ff7-93e0-495e-9d76-f8bce2783770 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:24.917393 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1385/2779] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:25.382323 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2c8d4-492e-449a-b0da-7ccb33d1c9d2 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:25.384504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-75b2c8d4-492e-449a-b0da-7ccb33d1c9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:25.397514 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2c8d4-492e-449a-b0da-7ccb33d1c9d2 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:25.406517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:25.407561 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:25.408055 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:25.408554 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] PATH_INFO: `/v3/projects/0742a538877c4743993c21d145c62ee1` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:25.410700 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0742a538877c4743993c21d145c62ee1)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:25.414336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:25.542162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0742a538877c4743993c21d145c62ee1', 'request_id': 'req-6e0add74-d23b-4abd-8c69-f53a5939f0ec'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:25.604280 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0742a538877c4743993c21d145c62ee1', 'request_id': 'req-6e0add74-d23b-4abd-8c69-f53a5939f0ec'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:25.610015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0742a538877c4743993c21d145c62ee1', 'request_id': 'req-6e0add74-d23b-4abd-8c69-f53a5939f0ec'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:25.622403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] The token cache is being invalidate because project 0742a538877c4743993c21d145c62ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:25.622694 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:25.623330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:25.624822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1395/2780] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:25 2022] DELETE /identity/v3/projects/0742a538877c4743993c21d145c62ee1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:25.628891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9ff7-93e0-495e-9d76-f8bce2783770 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:25.643727 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d9ff7-93e0-495e-9d76-f8bce2783770 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:25.659211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:25.704281 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:25.704898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:25.705167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:25.705434 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] PATH_INFO: `/v3/users/02bdc5bdd939453f8ca76c56282f302e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:25.706596 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=02bdc5bdd939453f8ca76c56282f302e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:25.708621 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:25.921373 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02bdc5bdd939453f8ca76c56282f302e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.026606 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02bdc5bdd939453f8ca76c56282f302e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.027205 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02bdc5bdd939453f8ca76c56282f302e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.034703 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02bdc5bdd939453f8ca76c56282f302e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.047588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02bdc5bdd939453f8ca76c56282f302e'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.053706 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1386/2781] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:25 2022] DELETE /identity/v3/users/02bdc5bdd939453f8ca76c56282f302e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:26.496704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:26.498190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:26.512158 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0add74-d23b-4abd-8c69-f53a5939f0ec admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:26.574592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:26.575260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:26.575543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:26.575817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] PATH_INFO: `/v3/projects/9cb828d5a46d42858cf80fe6ea329f06` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:26.577003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb828d5a46d42858cf80fe6ea329f06)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:26.578968 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:26.673692 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb828d5a46d42858cf80fe6ea329f06', 'request_id': 'req-8108f084-1124-4060-b0ed-ce888d46f3ac'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.735195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb828d5a46d42858cf80fe6ea329f06', 'request_id': 'req-8108f084-1124-4060-b0ed-ce888d46f3ac'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.739761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:26.742029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb828d5a46d42858cf80fe6ea329f06', 'request_id': 'req-8108f084-1124-4060-b0ed-ce888d46f3ac'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.754082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] The token cache is being invalidate because project 9cb828d5a46d42858cf80fe6ea329f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:26.754395 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.755032 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:26.756731 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1396/2782] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:26 2022] DELETE /identity/v3/projects/9cb828d5a46d42858cf80fe6ea329f06 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:26.761408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:26.762951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:26.776289 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c327a32-237e-41a3-a36e-f20031c86a2a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:26.836609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:26.837247 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:26.837526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:26.837800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] PATH_INFO: `/v3/users/9b11bf7b044c498caa5f30a22513ac8a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:26.838946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b11bf7b044c498caa5f30a22513ac8a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:26.863520 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:27.021129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:27.027206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b11bf7b044c498caa5f30a22513ac8a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.093092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b11bf7b044c498caa5f30a22513ac8a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.093729 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b11bf7b044c498caa5f30a22513ac8a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.103376 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b11bf7b044c498caa5f30a22513ac8a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.116397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b11bf7b044c498caa5f30a22513ac8a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.123139 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1387/2783] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:31:26 2022] DELETE /identity/v3/users/9b11bf7b044c498caa5f30a22513ac8a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:27.525647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:27.527276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:27.551265 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108f084-1124-4060-b0ed-ce888d46f3ac admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:27.615397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:27.616042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:27.616325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:27.616596 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] PATH_INFO: `/v3/projects/307b98abe5be4b789570a60e3d0d327a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:27.617742 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] RBAC: Authorizing `identity:delete_project(project_id=307b98abe5be4b789570a60e3d0d327a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:27.620276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:27.720543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '307b98abe5be4b789570a60e3d0d327a', 'request_id': 'req-383ed5d5-4215-4246-a9b2-d22294ea46be'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.776009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '307b98abe5be4b789570a60e3d0d327a', 'request_id': 'req-383ed5d5-4215-4246-a9b2-d22294ea46be'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.782193 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '307b98abe5be4b789570a60e3d0d327a', 'request_id': 'req-383ed5d5-4215-4246-a9b2-d22294ea46be'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.793757 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] The token cache is being invalidate because project 307b98abe5be4b789570a60e3d0d327a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:31:27.793976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.794625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-383ed5d5-4215-4246-a9b2-d22294ea46be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:31:27.796091 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1397/2784] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:31:27 2022] DELETE /identity/v3/projects/307b98abe5be4b789570a60e3d0d327a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:31:37.151441 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:DELETE=5,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:37.790633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=8,keystone:SELECT=55,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:31:45.185115 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:45.186594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:45.283691 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef2ee9-2897-4e6c-af3f-a5e8108aa2f4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:45.356899 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:45.357679 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:45.364083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:45.364083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:45.364083 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:45.374258 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:45.376223 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:45.389020 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1388/2785] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:47.226666 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ccff8-b7c9-4499-8aec-fa830776acc3 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:47.227143 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ccff8-b7c9-4499-8aec-fa830776acc3 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:47.227256 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ccff8-b7c9-4499-8aec-fa830776acc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:47.246272 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-fa2ccff8-b7c9-4499-8aec-fa830776acc3 None None] MFA Rules not processed for user `8c81c0af6f084b8db165fd51667b3b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:31:47.247512 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fa2ccff8-b7c9-4499-8aec-fa830776acc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:47.301564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1398/2786] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:31:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:31:47.628659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:47.630095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:47.685999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc3f2e-5676-455a-923b-147983c2ba79 service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:47.689731 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:47.690363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:47.690651 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:47.690914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:47.692623 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:47.706656 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:47.708057 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:47.720213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1389/2787] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:48.446708 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ccff8-b7c9-4499-8aec-fa830776acc3 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:48.448335 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fa2ccff8-b7c9-4499-8aec-fa830776acc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:48.520374 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ccff8-b7c9-4499-8aec-fa830776acc3 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:48.526466 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:48.527102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:48.527379 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:48.527649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:48.529393 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:48.541656 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:48.543354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:48.553381 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1399/2788] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:51.454904 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:31:51.456453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:51.467763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be9631c-c096-45b7-94ab-a791c5643189 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:31:51.473142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:31:51.473142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:31:51.473142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:31:51.473142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:31:51.474555 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:31:51.485540 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:31:51.487058 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:31:51.496097 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1390/2789] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:31:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:31:58.548932 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=57 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:32:01.491858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=94 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:32:02.935416 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:32:02.936003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:32:02.951372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05a33c-381e-4ed7-897e-77431c0d0682 service placement] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:32:02.956761 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5441541-b6d7-408a-87e9-122bcd4ab87d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:32:02.957343 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5441541-b6d7-408a-87e9-122bcd4ab87d service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:32:02.957617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5441541-b6d7-408a-87e9-122bcd4ab87d service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:32:02.957888 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5441541-b6d7-408a-87e9-122bcd4ab87d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:32:02.959681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b5441541-b6d7-408a-87e9-122bcd4ab87d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:32:02.971864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5441541-b6d7-408a-87e9-122bcd4ab87d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:32:02.973455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5441541-b6d7-408a-87e9-122bcd4ab87d service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:32:02.985001 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1400/2790] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:32:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:32:06.841502 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:32:06.855264 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea01f6f-c56e-49cf-a99e-73a25b6a6b04 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:32:06.860968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:32:06.861573 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:32:06.861858 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:32:06.862126 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:32:06.864167 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:32:06.877216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:32:06.878893 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:32:06.890443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1391/2791] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:32:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:32:12.980378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=6 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:32:16.885831 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=6 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:32:28.415316 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c316b4c-f095-4bc7-9238-091bef9846a4 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:32:28.415848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c316b4c-f095-4bc7-9238-091bef9846a4 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:32:28.415848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c316b4c-f095-4bc7-9238-091bef9846a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:32:28.436544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-5c316b4c-f095-4bc7-9238-091bef9846a4 None None] MFA Rules not processed for user `592da799f0c540d68a2cc6d54006467c`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:32:28.437739 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5c316b4c-f095-4bc7-9238-091bef9846a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:32:28.492123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1401/2792] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:32:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:32:28.797521 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:32:28.806760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41988a6a-7185-4b0a-a138-12abc8caf87e service nova] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:32:28.811447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeecab33-c81d-475c-9f76-e83393ea0731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:32:28.812031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecab33-c81d-475c-9f76-e83393ea0731 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:32:28.812303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecab33-c81d-475c-9f76-e83393ea0731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:32:28.812572 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeecab33-c81d-475c-9f76-e83393ea0731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:32:28.814290 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-aeecab33-c81d-475c-9f76-e83393ea0731 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:32:28.824604 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeecab33-c81d-475c-9f76-e83393ea0731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:32:28.826009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeecab33-c81d-475c-9f76-e83393ea0731 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:32:28.833891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1392/2793] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:32:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:32:38.485161 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=25 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:32:38.830324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=6 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:33:40.880229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c316b4c-f095-4bc7-9238-091bef9846a4 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:40.881610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-5c316b4c-f095-4bc7-9238-091bef9846a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:40.937746 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c316b4c-f095-4bc7-9238-091bef9846a4 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:40.941365 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:40.941967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:40.942246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:40.942548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:40.944253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:40.955278 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:40.956693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:40.966121 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1402/2794] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:33:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:33:43.295332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922e23f-08a8-495f-811d-b1f4d61a9dbf None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:43.295864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922e23f-08a8-495f-811d-b1f4d61a9dbf None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:43.295864 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922e23f-08a8-495f-811d-b1f4d61a9dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:43.315248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c922e23f-08a8-495f-811d-b1f4d61a9dbf None None] MFA Rules not processed for user `8c81c0af6f084b8db165fd51667b3b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:43.316305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c922e23f-08a8-495f-811d-b1f4d61a9dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:43.323694 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1393/2795] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:33:45.122060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:45.131463 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b92d61-e1b0-4c4a-b883-63bed400d14f service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:45.135889 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:45.136487 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:45.136766 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:45.137123 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:45.139462 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:45.205923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:45.207409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:45.217354 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1403/2796] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:33:49.066714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922e23f-08a8-495f-811d-b1f4d61a9dbf None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:49.068165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c922e23f-08a8-495f-811d-b1f4d61a9dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:49.099011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922e23f-08a8-495f-811d-b1f4d61a9dbf None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:49.107747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:49.109635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:49.109635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:49.109635 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:49.112361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:49.131197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:49.133040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:49.144178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1394/2797] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:33:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:33:49.207528 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:49.209575 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:49.225617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421609-f698-4bb2-87a2-0856c50859f0 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:49.231493 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:49.232120 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:49.235763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:49.235763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] PATH_INFO: `/v3/users/592da799f0c540d68a2cc6d54006467c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:49.235763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] RBAC: Authorizing `identity:delete_user(user_id=592da799f0c540d68a2cc6d54006467c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:49.235763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:49.398974 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592da799f0c540d68a2cc6d54006467c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.400275 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592da799f0c540d68a2cc6d54006467c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.452625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592da799f0c540d68a2cc6d54006467c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.466267 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592da799f0c540d68a2cc6d54006467c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.473276 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-918af920-ae41-4a18-ba22-60ae9cf5895d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592da799f0c540d68a2cc6d54006467c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.479454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1404/2798] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:33:49 2022] DELETE /identity/v3/users/592da799f0c540d68a2cc6d54006467c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:33:49.730274 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:49.739345 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd1ad8-6d5a-479b-845b-108cbf26eca5 service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:49.799730 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:49.800356 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:49.800632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:49.800928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] PATH_INFO: `/v3/projects/9d57c38134f1484d9b208de1b9d8e26e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:49.802037 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d57c38134f1484d9b208de1b9d8e26e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:49.804082 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:49.898311 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d57c38134f1484d9b208de1b9d8e26e', 'request_id': 'req-6dc50b44-4b60-4154-ae5d-d9473718c528'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.938443 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d57c38134f1484d9b208de1b9d8e26e', 'request_id': 'req-6dc50b44-4b60-4154-ae5d-d9473718c528'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.945107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d57c38134f1484d9b208de1b9d8e26e', 'request_id': 'req-6dc50b44-4b60-4154-ae5d-d9473718c528'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.957714 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] The token cache is being invalidate because project 9d57c38134f1484d9b208de1b9d8e26e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:33:49.958001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.958658 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-6dc50b44-4b60-4154-ae5d-d9473718c528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:49.960142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1395/2799] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:33:49 2022] DELETE /identity/v3/projects/9d57c38134f1484d9b208de1b9d8e26e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:33:49.965348 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5502380-3e10-40be-801a-5b59adc63b7a None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:49.965629 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5502380-3e10-40be-801a-5b59adc63b7a None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:49.965896 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5502380-3e10-40be-801a-5b59adc63b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:49.985259 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-b5502380-3e10-40be-801a-5b59adc63b7a None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:49.986419 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-b5502380-3e10-40be-801a-5b59adc63b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:49.990814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:33:50.046003 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1405/2800] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:33:50.053619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44009f-c5ee-43e9-9926-6676e04a70f6 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.053903 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44009f-c5ee-43e9-9926-6676e04a70f6 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.054173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f44009f-c5ee-43e9-9926-6676e04a70f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.074944 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-8f44009f-c5ee-43e9-9926-6676e04a70f6 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:50.076158 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-8f44009f-c5ee-43e9-9926-6676e04a70f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:50.084451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1396/2801] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:33:50.088676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5502380-3e10-40be-801a-5b59adc63b7a None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:50.098671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5502380-3e10-40be-801a-5b59adc63b7a None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:50.104288 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:50.105029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.105693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.106040 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.106794 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:33:50.107094 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:50.108662 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:50.121195 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1406/2802] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:33:50.125618 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44009f-c5ee-43e9-9926-6676e04a70f6 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:50.134818 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f44009f-c5ee-43e9-9926-6676e04a70f6 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:50.140093 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a325f1-24cd-4f27-b631-11f2fa408247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:50.140675 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a325f1-24cd-4f27-b631-11f2fa408247 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.140948 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a325f1-24cd-4f27-b631-11f2fa408247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.141211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a325f1-24cd-4f27-b631-11f2fa408247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.141836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a325f1-24cd-4f27-b631-11f2fa408247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:50.143742 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a325f1-24cd-4f27-b631-11f2fa408247 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:50.225065 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1397/2803] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:33:50 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:33:50.229782 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:50.240027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce1f2-7af1-4102-8946-0d6446963bf4 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:50.298523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010ce15-6855-4eaa-bc0b-a55073625ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:50.299137 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010ce15-6855-4eaa-bc0b-a55073625ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.299407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010ce15-6855-4eaa-bc0b-a55073625ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.299670 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010ce15-6855-4eaa-bc0b-a55073625ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.300344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010ce15-6855-4eaa-bc0b-a55073625ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:50.302233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010ce15-6855-4eaa-bc0b-a55073625ec8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:50.407521 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1407/2804] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:33:50 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:33:50.413851 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a325f1-24cd-4f27-b631-11f2fa408247 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:50.425468 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a325f1-24cd-4f27-b631-11f2fa408247 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:50.430471 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e3cda-95b7-4100-a1d3-90a1e24872af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:50.431209 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e3cda-95b7-4100-a1d3-90a1e24872af admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.431488 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e3cda-95b7-4100-a1d3-90a1e24872af admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.431757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e3cda-95b7-4100-a1d3-90a1e24872af admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.432480 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e3cda-95b7-4100-a1d3-90a1e24872af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:50.434531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e3cda-95b7-4100-a1d3-90a1e24872af admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:50.449090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1398/2805] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:33:50.453158 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010ce15-6855-4eaa-bc0b-a55073625ec8 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:50.464081 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010ce15-6855-4eaa-bc0b-a55073625ec8 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:50.468804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67e4ab-e0f4-4eb6-84ef-b3e45d4e09e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:50.469390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67e4ab-e0f4-4eb6-84ef-b3e45d4e09e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.469667 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67e4ab-e0f4-4eb6-84ef-b3e45d4e09e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.469942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67e4ab-e0f4-4eb6-84ef-b3e45d4e09e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.470600 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67e4ab-e0f4-4eb6-84ef-b3e45d4e09e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:50.472186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67e4ab-e0f4-4eb6-84ef-b3e45d4e09e8 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:50.481863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1408/2806] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:33:50.485816 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e3cda-95b7-4100-a1d3-90a1e24872af admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:50.496645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e3cda-95b7-4100-a1d3-90a1e24872af admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:50.501148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:50.501782 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.502054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.502318 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] PATH_INFO: `/v3/projects/05c30e8c72d6449e8defe028dc0b176b/users/275b3ea11e66495eacd10abbfb7fb0d2/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.517997 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c30e8c72d6449e8defe028dc0b176b, user_id=275b3ea11e66495eacd10abbfb7fb0d2, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:50.520659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:50.602013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1399/2807] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:33:50 2022] PUT /identity/v3/projects/05c30e8c72d6449e8defe028dc0b176b/users/275b3ea11e66495eacd10abbfb7fb0d2/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:33:50.612513 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f68f0b-3932-46cc-b531-4f31e5e91980 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.612799 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f68f0b-3932-46cc-b531-4f31e5e91980 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.613187 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f68f0b-3932-46cc-b531-4f31e5e91980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.644027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-65f68f0b-3932-46cc-b531-4f31e5e91980 None None] MFA Rules not processed for user `275b3ea11e66495eacd10abbfb7fb0d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:50.645241 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-65f68f0b-3932-46cc-b531-4f31e5e91980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:50.710803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1409/2808] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:33:50.723133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:50.724916 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:50.735023 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:33:50.799613 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9eb99-2fa0-407d-975c-3c448b9f26ae admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:50.805164 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a8326-143c-4433-8c9e-01aafb0786cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:50.806336 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a8326-143c-4433-8c9e-01aafb0786cb service nova] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:50.806687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a8326-143c-4433-8c9e-01aafb0786cb service nova] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:50.806959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a8326-143c-4433-8c9e-01aafb0786cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:50.815129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a8326-143c-4433-8c9e-01aafb0786cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:50.816741 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a8326-143c-4433-8c9e-01aafb0786cb service nova] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:50.826201 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1400/2809] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:33:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:33:53.890868 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f68f0b-3932-46cc-b531-4f31e5e91980 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:53.894254 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:33:53.951257 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f68f0b-3932-46cc-b531-4f31e5e91980 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:53.956660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:53.957289 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:53.957564 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:53.957831 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] PATH_INFO: `/v3/users/275b3ea11e66495eacd10abbfb7fb0d2` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:53.958945 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=275b3ea11e66495eacd10abbfb7fb0d2)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:53.960870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:54.152736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275b3ea11e66495eacd10abbfb7fb0d2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.153390 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275b3ea11e66495eacd10abbfb7fb0d2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.212447 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275b3ea11e66495eacd10abbfb7fb0d2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.225849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275b3ea11e66495eacd10abbfb7fb0d2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.233983 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275b3ea11e66495eacd10abbfb7fb0d2'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.240452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1410/2810] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:33:53 2022] DELETE /identity/v3/users/275b3ea11e66495eacd10abbfb7fb0d2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:33:54.244934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878401-16ef-4ea5-9640-e0f69c7cd467 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:54.245448 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878401-16ef-4ea5-9640-e0f69c7cd467 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:54.245554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb878401-16ef-4ea5-9640-e0f69c7cd467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:54.264239 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-bb878401-16ef-4ea5-9640-e0f69c7cd467 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:54.265433 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bb878401-16ef-4ea5-9640-e0f69c7cd467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:54.326988 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1401/2811] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:33:54.336848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:54.338904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:54.395483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b219a94-5490-4f7d-bcd3-686c9a8e7ebe admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:54.399029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e13c4-4a80-44dd-b587-32683a86cc48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:54.399620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e13c4-4a80-44dd-b587-32683a86cc48 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:54.399899 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e13c4-4a80-44dd-b587-32683a86cc48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:54.400165 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e13c4-4a80-44dd-b587-32683a86cc48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:54.409913 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e13c4-4a80-44dd-b587-32683a86cc48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:54.411358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e13c4-4a80-44dd-b587-32683a86cc48 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:54.423657 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1411/2812] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:33:54.808992 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb878401-16ef-4ea5-9640-e0f69c7cd467 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:54.813270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-bb878401-16ef-4ea5-9640-e0f69c7cd467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:54.824809 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb878401-16ef-4ea5-9640-e0f69c7cd467 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:54.829643 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:54.830300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:54.836453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:54.836453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] PATH_INFO: `/v3/projects/05c30e8c72d6449e8defe028dc0b176b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:54.836453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c30e8c72d6449e8defe028dc0b176b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:54.836453 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:54.933235 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c30e8c72d6449e8defe028dc0b176b', 'request_id': 'req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.972455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c30e8c72d6449e8defe028dc0b176b', 'request_id': 'req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.978117 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c30e8c72d6449e8defe028dc0b176b', 'request_id': 'req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.989234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] The token cache is being invalidate because project 05c30e8c72d6449e8defe028dc0b176b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:33:54.989519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.990428 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-99f27358-5a25-4598-a2b9-86e6e2c8fd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:33:54.991919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1402/2813] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:33:54 2022] DELETE /identity/v3/projects/05c30e8c72d6449e8defe028dc0b176b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:33:54.997315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244eec6-68bf-4659-bc79-5a1b52ac685d None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:54.997801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244eec6-68bf-4659-bc79-5a1b52ac685d None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:54.997927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244eec6-68bf-4659-bc79-5a1b52ac685d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.017714 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6244eec6-68bf-4659-bc79-5a1b52ac685d None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:55.018944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6244eec6-68bf-4659-bc79-5a1b52ac685d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:55.038774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:33:55.078659 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1412/2814] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:33:55.087707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440f08c-64ce-44d8-a784-95ebaccfb631 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.088016 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440f08c-64ce-44d8-a784-95ebaccfb631 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.088312 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440f08c-64ce-44d8-a784-95ebaccfb631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.107702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-f440f08c-64ce-44d8-a784-95ebaccfb631 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:55.108892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f440f08c-64ce-44d8-a784-95ebaccfb631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:55.117102 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1403/2815] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:33:55.121859 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244eec6-68bf-4659-bc79-5a1b52ac685d None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:55.131331 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244eec6-68bf-4659-bc79-5a1b52ac685d None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:55.136553 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:55.137139 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.137413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.137679 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.138314 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:33:55.138634 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:55.140208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:55.151760 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1413/2816] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:33:55.156160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440f08c-64ce-44d8-a784-95ebaccfb631 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:55.167815 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440f08c-64ce-44d8-a784-95ebaccfb631 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:55.174532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79e709-3dad-4643-a0cf-1415fb030ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:55.175133 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79e709-3dad-4643-a0cf-1415fb030ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.175411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79e709-3dad-4643-a0cf-1415fb030ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.175685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79e709-3dad-4643-a0cf-1415fb030ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.176308 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79e709-3dad-4643-a0cf-1415fb030ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:55.178200 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79e709-3dad-4643-a0cf-1415fb030ef7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:55.184319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:33:55.289523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1404/2817] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:33:55 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:33:55.294490 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:55.304504 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca414b-0701-479c-9aca-87c8f6c2e653 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:55.363569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427f1a-4986-4b5a-871b-a265c4fc9bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:55.364170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427f1a-4986-4b5a-871b-a265c4fc9bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.364444 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427f1a-4986-4b5a-871b-a265c4fc9bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.364981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427f1a-4986-4b5a-871b-a265c4fc9bbb admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.365684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427f1a-4986-4b5a-871b-a265c4fc9bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:55.367649 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427f1a-4986-4b5a-871b-a265c4fc9bbb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:55.430153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1414/2818] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:33:55 2022] POST /identity/v3/users => generated 282 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:33:55.435250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79e709-3dad-4643-a0cf-1415fb030ef7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:55.445536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79e709-3dad-4643-a0cf-1415fb030ef7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:55.450639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7192e-0e54-459b-8b28-0f1f20f08349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:55.451220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7192e-0e54-459b-8b28-0f1f20f08349 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.451499 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7192e-0e54-459b-8b28-0f1f20f08349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.451769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7192e-0e54-459b-8b28-0f1f20f08349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.452402 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7192e-0e54-459b-8b28-0f1f20f08349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:55.453983 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7192e-0e54-459b-8b28-0f1f20f08349 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:55.464048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1405/2819] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:33:55.468047 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427f1a-4986-4b5a-871b-a265c4fc9bbb admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:55.478092 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427f1a-4986-4b5a-871b-a265c4fc9bbb admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:55.483342 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15730c87-8406-4155-abd5-041bab5c45e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:55.483921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730c87-8406-4155-abd5-041bab5c45e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.484196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730c87-8406-4155-abd5-041bab5c45e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.484468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15730c87-8406-4155-abd5-041bab5c45e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.485182 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730c87-8406-4155-abd5-041bab5c45e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:55.486781 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15730c87-8406-4155-abd5-041bab5c45e5 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:55.498102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1415/2820] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 09 23:33:55.502296 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7192e-0e54-459b-8b28-0f1f20f08349 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:55.512735 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7192e-0e54-459b-8b28-0f1f20f08349 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:55.517381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9040e3c-8788-4ed6-bc98-216e77847023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:55.520421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9040e3c-8788-4ed6-bc98-216e77847023 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.520421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9040e3c-8788-4ed6-bc98-216e77847023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.520421 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9040e3c-8788-4ed6-bc98-216e77847023 admin admin] PATH_INFO: `/v3/projects/1aea9c0233004aaabc37668c1a54d37b/users/969f4cc0c8f544cf9fc98d8d622c65ee/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.533342 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9040e3c-8788-4ed6-bc98-216e77847023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aea9c0233004aaabc37668c1a54d37b, user_id=969f4cc0c8f544cf9fc98d8d622c65ee, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:55.536081 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9040e3c-8788-4ed6-bc98-216e77847023 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:55.619095 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1406/2821] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:33:55 2022] PUT /identity/v3/projects/1aea9c0233004aaabc37668c1a54d37b/users/969f4cc0c8f544cf9fc98d8d622c65ee/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:33:55.630803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff9890-58ac-49ed-bd95-9befe40bb528 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.631115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff9890-58ac-49ed-bd95-9befe40bb528 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.631413 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff9890-58ac-49ed-bd95-9befe40bb528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.659296 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-7cff9890-58ac-49ed-bd95-9befe40bb528 None None] MFA Rules not processed for user `969f4cc0c8f544cf9fc98d8d622c65ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:55.660574 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-7cff9890-58ac-49ed-bd95-9befe40bb528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:55.725867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1416/2822] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:33:55.730912 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9040e3c-8788-4ed6-bc98-216e77847023 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:55.801759 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9040e3c-8788-4ed6-bc98-216e77847023 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:55.806469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29320066-4a39-4752-9951-de659f9ce798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:55.807070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29320066-4a39-4752-9951-de659f9ce798 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.807398 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29320066-4a39-4752-9951-de659f9ce798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.808216 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29320066-4a39-4752-9951-de659f9ce798 admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.808848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29320066-4a39-4752-9951-de659f9ce798 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:55.810787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29320066-4a39-4752-9951-de659f9ce798 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:55.910363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1407/2823] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:33:55 2022] POST /identity/v3/projects => generated 363 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:33:55.914602 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff9890-58ac-49ed-bd95-9befe40bb528 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:55.926063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff9890-58ac-49ed-bd95-9befe40bb528 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:55.943988 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:33:55.984542 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a851e-3462-4ab7-a96a-a7543037c2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:55.985063 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a851e-3462-4ab7-a96a-a7543037c2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:55.985338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a851e-3462-4ab7-a96a-a7543037c2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:55.985605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a851e-3462-4ab7-a96a-a7543037c2b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:55.986229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a851e-3462-4ab7-a96a-a7543037c2b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:55.988171 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a851e-3462-4ab7-a96a-a7543037c2b6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:56.086548 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1417/2824] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:33:55 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:33:56.091255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29320066-4a39-4752-9951-de659f9ce798 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:56.104960 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29320066-4a39-4752-9951-de659f9ce798 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:56.109769 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90097251-37ba-4b97-bd47-f75a3cba70a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:56.110451 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90097251-37ba-4b97-bd47-f75a3cba70a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:56.110732 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90097251-37ba-4b97-bd47-f75a3cba70a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:56.111001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90097251-37ba-4b97-bd47-f75a3cba70a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:56.111638 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90097251-37ba-4b97-bd47-f75a3cba70a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:56.113229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90097251-37ba-4b97-bd47-f75a3cba70a0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:56.125332 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1408/2825] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:33:56.129863 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a851e-3462-4ab7-a96a-a7543037c2b6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:56.141329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a851e-3462-4ab7-a96a-a7543037c2b6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:56.146183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1e73f-9d1a-4196-8a63-32568db089e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:56.146869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1e73f-9d1a-4196-8a63-32568db089e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:56.147146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1e73f-9d1a-4196-8a63-32568db089e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:56.147415 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1e73f-9d1a-4196-8a63-32568db089e2 admin admin] PATH_INFO: `/v3/projects/8cec8f876aa043fe9b2c41a6749fb314/users/834f0d16c3db426794350f4c2d5b050b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:56.163279 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1e73f-9d1a-4196-8a63-32568db089e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cec8f876aa043fe9b2c41a6749fb314, user_id=834f0d16c3db426794350f4c2d5b050b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:56.165921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1e73f-9d1a-4196-8a63-32568db089e2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:56.232607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1418/2826] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:33:56 2022] PUT /identity/v3/projects/8cec8f876aa043fe9b2c41a6749fb314/users/834f0d16c3db426794350f4c2d5b050b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:33:56.237054 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90097251-37ba-4b97-bd47-f75a3cba70a0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:56.248400 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90097251-37ba-4b97-bd47-f75a3cba70a0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:56.305909 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6236b95-e44d-4762-a78d-f56f5ee8bb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:56.306645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236b95-e44d-4762-a78d-f56f5ee8bb12 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:56.306797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236b95-e44d-4762-a78d-f56f5ee8bb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:56.307060 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6236b95-e44d-4762-a78d-f56f5ee8bb12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:56.307996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6236b95-e44d-4762-a78d-f56f5ee8bb12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:56.309754 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6236b95-e44d-4762-a78d-f56f5ee8bb12 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:56.320114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1409/2827] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:33:56.324862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1e73f-9d1a-4196-8a63-32568db089e2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:56.335370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1e73f-9d1a-4196-8a63-32568db089e2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:56.340697 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:56.341338 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:56.341617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:56.341886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] PATH_INFO: `/v3/projects/8cec8f876aa043fe9b2c41a6749fb314/users/834f0d16c3db426794350f4c2d5b050b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:56.343902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cec8f876aa043fe9b2c41a6749fb314, user_id=834f0d16c3db426794350f4c2d5b050b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:56.346604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:56.356800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1419/2828] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:33:56 2022] PUT /identity/v3/projects/8cec8f876aa043fe9b2c41a6749fb314/users/834f0d16c3db426794350f4c2d5b050b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:33:56.367226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e46b4-c753-4523-ab32-09c91720b875 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:56.367513 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e46b4-c753-4523-ab32-09c91720b875 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:56.367789 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e46b4-c753-4523-ab32-09c91720b875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:56.395723 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-313e46b4-c753-4523-ab32-09c91720b875 None None] MFA Rules not processed for user `834f0d16c3db426794350f4c2d5b050b`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:33:56.396925 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-313e46b4-c753-4523-ab32-09c91720b875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:56.407687 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:33:56.470175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1410/2829] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:33:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:33:56.481461 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:56.483371 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:33:56.544068 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc3164-7b18-48b8-ba94-e7fd8686a8be admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:56.547852 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:56.548456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:56.548733 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:56.549129 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:56.559961 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:56.561399 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:56.570981 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1420/2830] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:33:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:33:56.728822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e46b4-c753-4523-ab32-09c91720b875 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:33:56.739070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e46b4-c753-4523-ab32-09c91720b875 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:33:56.743510 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0b48-6e17-499b-bd3a-e98a614fb9cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:33:56.744125 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0b48-6e17-499b-bd3a-e98a614fb9cc service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:33:56.744444 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0b48-6e17-499b-bd3a-e98a614fb9cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:33:56.744772 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff0b48-6e17-499b-bd3a-e98a614fb9cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:33:56.801907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff0b48-6e17-499b-bd3a-e98a614fb9cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:33:56.803361 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff0b48-6e17-499b-bd3a-e98a614fb9cc service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:33:56.813048 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1411/2831] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:33:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:03.475562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:03.477501 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:03.558921 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce734fe-fd9e-4864-85a8-5e0072529af1 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:03.564344 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381ac42-77a4-41d2-83a7-14efbc09bdb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3bec982c8474c349ca4901039844d26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:03.564947 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381ac42-77a4-41d2-83a7-14efbc09bdb4 service nova] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:03.565221 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381ac42-77a4-41d2-83a7-14efbc09bdb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:03.565486 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381ac42-77a4-41d2-83a7-14efbc09bdb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:03.567202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-e381ac42-77a4-41d2-83a7-14efbc09bdb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:03.580255 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:03.625308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381ac42-77a4-41d2-83a7-14efbc09bdb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:03.626915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381ac42-77a4-41d2-83a7-14efbc09bdb4 service nova] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:03.635762 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1421/2832] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:06.808913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=52 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:08.562478 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0b48-6e17-499b-bd3a-e98a614fb9cc service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:08.624447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff0b48-6e17-499b-bd3a-e98a614fb9cc service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:08.628869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:08.629487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:08.629761 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:08.630032 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] PATH_INFO: `/v3/users/969f4cc0c8f544cf9fc98d8d622c65ee` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:08.631173 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=969f4cc0c8f544cf9fc98d8d622c65ee)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:08.633094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:08.775245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '969f4cc0c8f544cf9fc98d8d622c65ee'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:08.818194 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '969f4cc0c8f544cf9fc98d8d622c65ee'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:08.818811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '969f4cc0c8f544cf9fc98d8d622c65ee'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:08.826805 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '969f4cc0c8f544cf9fc98d8d622c65ee'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:08.838786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '969f4cc0c8f544cf9fc98d8d622c65ee'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:08.845693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1412/2833] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:34:08 2022] DELETE /identity/v3/users/969f4cc0c8f544cf9fc98d8d622c65ee => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:08.850166 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d1ace-0c1c-4e30-b5b3-7e333fade9bc None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:08.850729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d1ace-0c1c-4e30-b5b3-7e333fade9bc None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:08.850729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d1ace-0c1c-4e30-b5b3-7e333fade9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:08.869737 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-503d1ace-0c1c-4e30-b5b3-7e333fade9bc None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:08.870964 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-503d1ace-0c1c-4e30-b5b3-7e333fade9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:08.929410 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1422/2834] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:08.939128 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:08.940523 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:08.997996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325327c9-15f5-40b9-8e8e-28c4ffd4d439 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:09.001605 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453752e-950f-4588-bc59-2ca1ebdf1f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:09.002251 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453752e-950f-4588-bc59-2ca1ebdf1f32 service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:09.002554 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453752e-950f-4588-bc59-2ca1ebdf1f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:09.002820 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453752e-950f-4588-bc59-2ca1ebdf1f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:09.011718 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453752e-950f-4588-bc59-2ca1ebdf1f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:09.013747 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453752e-950f-4588-bc59-2ca1ebdf1f32 service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:09.023246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1413/2835] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:09.420726 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d1ace-0c1c-4e30-b5b3-7e333fade9bc None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:09.428588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-503d1ace-0c1c-4e30-b5b3-7e333fade9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:09.435955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d1ace-0c1c-4e30-b5b3-7e333fade9bc None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:09.441311 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:09.441884 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:09.442231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:09.442543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] PATH_INFO: `/v3/projects/1aea9c0233004aaabc37668c1a54d37b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:09.443664 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aea9c0233004aaabc37668c1a54d37b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:09.445704 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:09.532360 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aea9c0233004aaabc37668c1a54d37b', 'request_id': 'req-29ccc064-50a8-48ed-bbe1-288a867dc1cf'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.578127 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aea9c0233004aaabc37668c1a54d37b', 'request_id': 'req-29ccc064-50a8-48ed-bbe1-288a867dc1cf'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.583815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aea9c0233004aaabc37668c1a54d37b', 'request_id': 'req-29ccc064-50a8-48ed-bbe1-288a867dc1cf'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.595325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] The token cache is being invalidate because project 1aea9c0233004aaabc37668c1a54d37b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:34:09.595606 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.596224 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.597693 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1423/2836] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:34:09 2022] DELETE /identity/v3/projects/1aea9c0233004aaabc37668c1a54d37b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:09.601913 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453752e-950f-4588-bc59-2ca1ebdf1f32 service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:09.612182 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453752e-950f-4588-bc59-2ca1ebdf1f32 service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:09.630832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:09.668150 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:09.668763 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:09.669040 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:09.669305 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] PATH_INFO: `/v3/users/834f0d16c3db426794350f4c2d5b050b` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:09.671005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=834f0d16c3db426794350f4c2d5b050b)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:09.673022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:09.872660 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '834f0d16c3db426794350f4c2d5b050b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.948987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '834f0d16c3db426794350f4c2d5b050b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.949611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '834f0d16c3db426794350f4c2d5b050b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.962411 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '834f0d16c3db426794350f4c2d5b050b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.977844 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-0a5da9af-5e07-406f-86b8-88f3226951f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '834f0d16c3db426794350f4c2d5b050b'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:09.984639 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1414/2837] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:34:09 2022] DELETE /identity/v3/users/834f0d16c3db426794350f4c2d5b050b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:10.363170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:10.365268 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:10.383307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ccc064-50a8-48ed-bbe1-288a867dc1cf admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:10.417847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:10.439923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:10.440555 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:10.440824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:10.441074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] PATH_INFO: `/v3/projects/8cec8f876aa043fe9b2c41a6749fb314` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:10.442159 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cec8f876aa043fe9b2c41a6749fb314)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:10.444102 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:10.555072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cec8f876aa043fe9b2c41a6749fb314', 'request_id': 'req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:10.629251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cec8f876aa043fe9b2c41a6749fb314', 'request_id': 'req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:10.634937 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cec8f876aa043fe9b2c41a6749fb314', 'request_id': 'req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:10.647184 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] The token cache is being invalidate because project 8cec8f876aa043fe9b2c41a6749fb314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:34:10.647467 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:10.648079 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-bf6913c0-c65e-4cf1-bd2b-8a38bd2dfeb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:10.649588 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1424/2838] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:34:10 2022] DELETE /identity/v3/projects/8cec8f876aa043fe9b2c41a6749fb314 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:34:10.655009 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b6221-575e-47fc-8d3c-83bb66460976 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:10.655526 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b6221-575e-47fc-8d3c-83bb66460976 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:10.655647 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b6221-575e-47fc-8d3c-83bb66460976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:10.675174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-995b6221-575e-47fc-8d3c-83bb66460976 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:10.676278 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-995b6221-575e-47fc-8d3c-83bb66460976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:10.734984 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1415/2839] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:10.743148 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d294c75-2ea3-4208-b2b6-059dbf2b5481 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:10.743433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d294c75-2ea3-4208-b2b6-059dbf2b5481 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:10.743709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d294c75-2ea3-4208-b2b6-059dbf2b5481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:10.764797 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-8d294c75-2ea3-4208-b2b6-059dbf2b5481 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:10.766000 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-8d294c75-2ea3-4208-b2b6-059dbf2b5481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:10.773919 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1425/2840] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:10.778211 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b6221-575e-47fc-8d3c-83bb66460976 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:10.788001 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b6221-575e-47fc-8d3c-83bb66460976 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:10.792515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:10.793101 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:10.793408 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:10.793676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:10.794309 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:34:10.794632 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:10.796255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:10.808834 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1416/2841] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:34:10.814067 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d294c75-2ea3-4208-b2b6-059dbf2b5481 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:10.823421 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d294c75-2ea3-4208-b2b6-059dbf2b5481 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:10.829018 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f803055-c23d-40b6-9c7e-1f70c2183262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:10.829617 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f803055-c23d-40b6-9c7e-1f70c2183262 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:10.829920 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f803055-c23d-40b6-9c7e-1f70c2183262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:10.830190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f803055-c23d-40b6-9c7e-1f70c2183262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:10.830865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f803055-c23d-40b6-9c7e-1f70c2183262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:10.832758 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f803055-c23d-40b6-9c7e-1f70c2183262 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:10.893598 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1426/2842] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:34:10 2022] POST /identity/v3/projects => generated 387 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:10.898077 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:10.908783 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0d3bb-e73d-4db1-8b65-e352e9dded24 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:10.934584 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:10.964669 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5e56f-0914-42ac-b97c-5760eb09e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:10.965269 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5e56f-0914-42ac-b97c-5760eb09e569 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:10.965545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5e56f-0914-42ac-b97c-5760eb09e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:10.965814 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5e56f-0914-42ac-b97c-5760eb09e569 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:10.966474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5e56f-0914-42ac-b97c-5760eb09e569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:10.968883 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5e56f-0914-42ac-b97c-5760eb09e569 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.035343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1417/2843] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:34:10 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:11.039479 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f803055-c23d-40b6-9c7e-1f70c2183262 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:11.050115 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f803055-c23d-40b6-9c7e-1f70c2183262 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:11.054776 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4185b7-c10b-44f0-bf18-c59338f144e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:11.055370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4185b7-c10b-44f0-bf18-c59338f144e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.055645 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4185b7-c10b-44f0-bf18-c59338f144e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.055914 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4185b7-c10b-44f0-bf18-c59338f144e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.056578 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4185b7-c10b-44f0-bf18-c59338f144e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:11.058158 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4185b7-c10b-44f0-bf18-c59338f144e0 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.068368 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1427/2844] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:11.072132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5e56f-0914-42ac-b97c-5760eb09e569 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:11.083106 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5e56f-0914-42ac-b97c-5760eb09e569 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:11.087891 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13f748-1e41-4dd3-87b5-c3584461ddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:11.088491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f748-1e41-4dd3-87b5-c3584461ddcb admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.088770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f748-1e41-4dd3-87b5-c3584461ddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.089044 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f748-1e41-4dd3-87b5-c3584461ddcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.089683 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13f748-1e41-4dd3-87b5-c3584461ddcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:11.091358 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13f748-1e41-4dd3-87b5-c3584461ddcb admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.101922 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1418/2845] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:11.105976 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4185b7-c10b-44f0-bf18-c59338f144e0 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:11.116330 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4185b7-c10b-44f0-bf18-c59338f144e0 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:11.120892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e9cbc-b9a9-4b6e-bad4-d0f4958a1b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:11.121526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e9cbc-b9a9-4b6e-bad4-d0f4958a1b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.121801 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e9cbc-b9a9-4b6e-bad4-d0f4958a1b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.122072 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e9cbc-b9a9-4b6e-bad4-d0f4958a1b0f admin admin] PATH_INFO: `/v3/projects/a70d5a50b851465d88ed85969f9d37d3/users/0d85147715d64a40bd2aa708af6ef7c7/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.136336 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e9cbc-b9a9-4b6e-bad4-d0f4958a1b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a70d5a50b851465d88ed85969f9d37d3, user_id=0d85147715d64a40bd2aa708af6ef7c7, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:11.139530 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e9cbc-b9a9-4b6e-bad4-d0f4958a1b0f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.252709 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1428/2846] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:11 2022] PUT /identity/v3/projects/a70d5a50b851465d88ed85969f9d37d3/users/0d85147715d64a40bd2aa708af6ef7c7/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:11.262861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22f65d-3c5c-4836-94c6-f345d996e57a None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.263148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22f65d-3c5c-4836-94c6-f345d996e57a None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.263447 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22f65d-3c5c-4836-94c6-f345d996e57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.292994 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-5b22f65d-3c5c-4836-94c6-f345d996e57a None None] MFA Rules not processed for user `0d85147715d64a40bd2aa708af6ef7c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:11.294178 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5b22f65d-3c5c-4836-94c6-f345d996e57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:11.356366 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1419/2847] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:11.360935 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e9cbc-b9a9-4b6e-bad4-d0f4958a1b0f admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:11.422261 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e9cbc-b9a9-4b6e-bad4-d0f4958a1b0f admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:11.426909 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e2663-ff56-41f1-9ca3-8f265e0ea4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:11.427508 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e2663-ff56-41f1-9ca3-8f265e0ea4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.427784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e2663-ff56-41f1-9ca3-8f265e0ea4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.428053 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e2663-ff56-41f1-9ca3-8f265e0ea4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.428675 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e2663-ff56-41f1-9ca3-8f265e0ea4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:11.430572 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e2663-ff56-41f1-9ca3-8f265e0ea4c7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.440870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:11.505339 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1429/2848] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:34:11 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:11.509363 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22f65d-3c5c-4836-94c6-f345d996e57a None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:11.521226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22f65d-3c5c-4836-94c6-f345d996e57a None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:11.577515 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285424c-eb91-4b8f-a9d6-dfa032ba29f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:11.578399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285424c-eb91-4b8f-a9d6-dfa032ba29f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.578685 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285424c-eb91-4b8f-a9d6-dfa032ba29f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.578959 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285424c-eb91-4b8f-a9d6-dfa032ba29f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.579599 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285424c-eb91-4b8f-a9d6-dfa032ba29f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:11.581519 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285424c-eb91-4b8f-a9d6-dfa032ba29f7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.657693 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1420/2849] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:34:11 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:11.661481 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e2663-ff56-41f1-9ca3-8f265e0ea4c7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:11.672442 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e2663-ff56-41f1-9ca3-8f265e0ea4c7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:11.678253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a50a6a-5733-42fd-9dfa-a820e640a6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:11.678992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a50a6a-5733-42fd-9dfa-a820e640a6eb admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.679248 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a50a6a-5733-42fd-9dfa-a820e640a6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.679590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a50a6a-5733-42fd-9dfa-a820e640a6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.680223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a50a6a-5733-42fd-9dfa-a820e640a6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:11.681870 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a50a6a-5733-42fd-9dfa-a820e640a6eb admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.692880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1430/2850] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:11.696869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285424c-eb91-4b8f-a9d6-dfa032ba29f7 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:11.707876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285424c-eb91-4b8f-a9d6-dfa032ba29f7 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:11.712491 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615dd017-accc-4fa6-8b7b-448c72013674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:11.713107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615dd017-accc-4fa6-8b7b-448c72013674 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.713381 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615dd017-accc-4fa6-8b7b-448c72013674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.713649 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615dd017-accc-4fa6-8b7b-448c72013674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.714277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615dd017-accc-4fa6-8b7b-448c72013674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:11.715910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615dd017-accc-4fa6-8b7b-448c72013674 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.725767 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1421/2851] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:11.729663 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a50a6a-5733-42fd-9dfa-a820e640a6eb admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:11.741074 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a50a6a-5733-42fd-9dfa-a820e640a6eb admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:11.745751 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853ec4a-8900-4e2d-a12f-351cc7e0caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:11.746452 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ec4a-8900-4e2d-a12f-351cc7e0caaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.746732 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ec4a-8900-4e2d-a12f-351cc7e0caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.747004 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853ec4a-8900-4e2d-a12f-351cc7e0caaa admin admin] PATH_INFO: `/v3/projects/b0246d177bcf409f8ee630e80e72e967/users/a2e2d2bf899e41db9d725c6fea70e768/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.762050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853ec4a-8900-4e2d-a12f-351cc7e0caaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0246d177bcf409f8ee630e80e72e967, user_id=a2e2d2bf899e41db9d725c6fea70e768, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:11.764862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853ec4a-8900-4e2d-a12f-351cc7e0caaa admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:11.831205 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1431/2852] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:11 2022] PUT /identity/v3/projects/b0246d177bcf409f8ee630e80e72e967/users/a2e2d2bf899e41db9d725c6fea70e768/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:11.896382 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515073-294a-41f6-bbcf-dac53a42e04b None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:11.896670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515073-294a-41f6-bbcf-dac53a42e04b None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:11.896977 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515073-294a-41f6-bbcf-dac53a42e04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:11.925319 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-82515073-294a-41f6-bbcf-dac53a42e04b None None] MFA Rules not processed for user `a2e2d2bf899e41db9d725c6fea70e768`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:11.926567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-82515073-294a-41f6-bbcf-dac53a42e04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:11.930430 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:11.993034 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1422/2853] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:34:11.997491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853ec4a-8900-4e2d-a12f-351cc7e0caaa admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:12.057109 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853ec4a-8900-4e2d-a12f-351cc7e0caaa admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:12.062154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341f6c2-d5b5-403d-bdc1-102b4166e44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:12.062770 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341f6c2-d5b5-403d-bdc1-102b4166e44b admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:12.063041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341f6c2-d5b5-403d-bdc1-102b4166e44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:12.063306 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341f6c2-d5b5-403d-bdc1-102b4166e44b admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:12.063923 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341f6c2-d5b5-403d-bdc1-102b4166e44b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:12.065788 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341f6c2-d5b5-403d-bdc1-102b4166e44b admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:12.138632 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1432/2854] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:34:11 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:12.142797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82515073-294a-41f6-bbcf-dac53a42e04b None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:12.153808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82515073-294a-41f6-bbcf-dac53a42e04b None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:12.210154 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636ac4d-acbe-4ffe-80c7-9a36b4c52701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:12.210790 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636ac4d-acbe-4ffe-80c7-9a36b4c52701 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:12.211069 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636ac4d-acbe-4ffe-80c7-9a36b4c52701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:12.211341 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636ac4d-acbe-4ffe-80c7-9a36b4c52701 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:12.211978 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636ac4d-acbe-4ffe-80c7-9a36b4c52701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:12.213878 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636ac4d-acbe-4ffe-80c7-9a36b4c52701 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:12.315715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1423/2855] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:34:12 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:12.319687 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341f6c2-d5b5-403d-bdc1-102b4166e44b admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:12.330126 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341f6c2-d5b5-403d-bdc1-102b4166e44b admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:12.334741 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d830d-25fa-4d7f-b129-d3cd6e00edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:12.335328 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d830d-25fa-4d7f-b129-d3cd6e00edbc admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:12.335604 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d830d-25fa-4d7f-b129-d3cd6e00edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:12.335873 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d830d-25fa-4d7f-b129-d3cd6e00edbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:12.336509 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d830d-25fa-4d7f-b129-d3cd6e00edbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:12.338080 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d830d-25fa-4d7f-b129-d3cd6e00edbc admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:12.350251 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1433/2856] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:12.354467 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636ac4d-acbe-4ffe-80c7-9a36b4c52701 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:12.365609 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636ac4d-acbe-4ffe-80c7-9a36b4c52701 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:12.370253 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695b54f-2902-4239-8423-5018044dd6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:12.370919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695b54f-2902-4239-8423-5018044dd6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:12.371195 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695b54f-2902-4239-8423-5018044dd6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:12.371464 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695b54f-2902-4239-8423-5018044dd6d0 admin admin] PATH_INFO: `/v3/projects/8d50ed1146bf4c499fafefbe6ff3de05/users/e7c581ddfb954607aea27232f26f4523/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:12.386770 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695b54f-2902-4239-8423-5018044dd6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d50ed1146bf4c499fafefbe6ff3de05, user_id=e7c581ddfb954607aea27232f26f4523, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:12.389469 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4695b54f-2902-4239-8423-5018044dd6d0 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:12.458588 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1424/2857] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:12 2022] PUT /identity/v3/projects/8d50ed1146bf4c499fafefbe6ff3de05/users/e7c581ddfb954607aea27232f26f4523/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:12.462625 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d830d-25fa-4d7f-b129-d3cd6e00edbc admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:12.473612 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d830d-25fa-4d7f-b129-d3cd6e00edbc admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:12.533764 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36ea64-caa3-42f4-a850-41ec8a84f24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:12.534385 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36ea64-caa3-42f4-a850-41ec8a84f24f admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:12.534668 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36ea64-caa3-42f4-a850-41ec8a84f24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:12.534938 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36ea64-caa3-42f4-a850-41ec8a84f24f admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:12.535562 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36ea64-caa3-42f4-a850-41ec8a84f24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:12.537134 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36ea64-caa3-42f4-a850-41ec8a84f24f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:12.547146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1434/2858] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:12.551536 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695b54f-2902-4239-8423-5018044dd6d0 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:12.562288 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695b54f-2902-4239-8423-5018044dd6d0 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:12.566950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:12.567594 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:12.567872 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:12.568147 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] PATH_INFO: `/v3/projects/8d50ed1146bf4c499fafefbe6ff3de05/users/e7c581ddfb954607aea27232f26f4523/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:12.570055 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d50ed1146bf4c499fafefbe6ff3de05, user_id=e7c581ddfb954607aea27232f26f4523, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:12.572725 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:12.583655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1425/2859] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:12 2022] PUT /identity/v3/projects/8d50ed1146bf4c499fafefbe6ff3de05/users/e7c581ddfb954607aea27232f26f4523/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:12.594317 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d869d4e-89a8-46e8-aa26-58dabe910d67 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:12.594705 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d869d4e-89a8-46e8-aa26-58dabe910d67 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:12.594982 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d869d4e-89a8-46e8-aa26-58dabe910d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:12.622485 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-9d869d4e-89a8-46e8-aa26-58dabe910d67 None None] MFA Rules not processed for user `e7c581ddfb954607aea27232f26f4523`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:12.623977 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9d869d4e-89a8-46e8-aa26-58dabe910d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:12.635401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:12.691408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1435/2860] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:34:12.703496 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:12.705213 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:12.763270 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a99625-182a-4fd8-b638-0544b253e1e8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:12.767076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94c2cf-667c-4e33-8674-7493c7c10633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:12.767676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94c2cf-667c-4e33-8674-7493c7c10633 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:12.767950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94c2cf-667c-4e33-8674-7493c7c10633 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:12.768220 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94c2cf-667c-4e33-8674-7493c7c10633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:12.777873 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94c2cf-667c-4e33-8674-7493c7c10633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:12.779422 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94c2cf-667c-4e33-8674-7493c7c10633 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:12.787166 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:12.789249 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1426/2861] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:13.906681 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d869d4e-89a8-46e8-aa26-58dabe910d67 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:13.908397 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-9d869d4e-89a8-46e8-aa26-58dabe910d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:13.921691 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d869d4e-89a8-46e8-aa26-58dabe910d67 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:13.925326 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382a22b-f11a-4e00-9e04-134747472754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:13.925924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a22b-f11a-4e00-9e04-134747472754 service cinder] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:13.926204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a22b-f11a-4e00-9e04-134747472754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:13.926511 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a22b-f11a-4e00-9e04-134747472754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:13.986483 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382a22b-f11a-4e00-9e04-134747472754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:13.987846 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382a22b-f11a-4e00-9e04-134747472754 service cinder] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:13.996911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1436/2862] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:18.862218 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94c2cf-667c-4e33-8674-7493c7c10633 service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:18.871371 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94c2cf-667c-4e33-8674-7493c7c10633 service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:18.876011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:18.876593 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:18.879652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:18.879652 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:18.933348 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:18.934824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:18.944410 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1427/2863] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:22.773734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382a22b-f11a-4e00-9e04-134747472754 service cinder] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:22.836713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382a22b-f11a-4e00-9e04-134747472754 service cinder] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:22.841327 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:22.842002 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:22.842370 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:22.842655 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] PATH_INFO: `/v3/users/0d85147715d64a40bd2aa708af6ef7c7` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:22.843803 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d85147715d64a40bd2aa708af6ef7c7)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:22.845654 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:22.984498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:22.985934 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:22.988772 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d85147715d64a40bd2aa708af6ef7c7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:22.989747 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d85147715d64a40bd2aa708af6ef7c7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:22.992835 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:23.057757 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2053fc-c926-4f2c-9917-9f850f0df89b service cinder] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:23.058644 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d85147715d64a40bd2aa708af6ef7c7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:23.063387 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:23.064073 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:23.064351 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:23.068328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:23.068328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:23.072907 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d85147715d64a40bd2aa708af6ef7c7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:23.079250 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:23.079844 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-8c2c2f2c-79eb-4959-a78e-e50d50abf54f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d85147715d64a40bd2aa708af6ef7c7'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:23.080839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:23.085723 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1437/2864] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:34:22 2022] DELETE /identity/v3/users/0d85147715d64a40bd2aa708af6ef7c7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:23.090071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e3e88-8170-44fa-9071-7d1e5ddfa9ab None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:23.090387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e3e88-8170-44fa-9071-7d1e5ddfa9ab None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:23.090669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e3e88-8170-44fa-9071-7d1e5ddfa9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:23.091148 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1428/2865] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:34:23.110295 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-230e3e88-8170-44fa-9071-7d1e5ddfa9ab None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:23.111549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-230e3e88-8170-44fa-9071-7d1e5ddfa9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:23.171013 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1438/2866] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 09 23:34:23.181329 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:23.182910 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:23.240929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce994f24-ddbb-4a47-acb3-e753a6bf0e05 service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:23.282597 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=100 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:23.289964 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de5dc8-60b8-4cee-9f2d-5dc4b99192bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:23.290645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de5dc8-60b8-4cee-9f2d-5dc4b99192bd service neutron] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:23.290918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de5dc8-60b8-4cee-9f2d-5dc4b99192bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:23.291180 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de5dc8-60b8-4cee-9f2d-5dc4b99192bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:23.300919 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de5dc8-60b8-4cee-9f2d-5dc4b99192bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:23.302394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de5dc8-60b8-4cee-9f2d-5dc4b99192bd service neutron] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:23.312786 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1429/2867] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:23.705260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e3e88-8170-44fa-9071-7d1e5ddfa9ab None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:23.706867 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-230e3e88-8170-44fa-9071-7d1e5ddfa9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:23.724387 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e3e88-8170-44fa-9071-7d1e5ddfa9ab None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:23.729915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:23.730729 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:23.730849 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:23.731103 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] PATH_INFO: `/v3/projects/a70d5a50b851465d88ed85969f9d37d3` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:23.733051 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a70d5a50b851465d88ed85969f9d37d3)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:23.735611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:23.831089 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a70d5a50b851465d88ed85969f9d37d3', 'request_id': 'req-6d5b160e-2bd6-443f-849d-2f0857853f12'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:23.921427 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a70d5a50b851465d88ed85969f9d37d3', 'request_id': 'req-6d5b160e-2bd6-443f-849d-2f0857853f12'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:23.929647 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a70d5a50b851465d88ed85969f9d37d3', 'request_id': 'req-6d5b160e-2bd6-443f-849d-2f0857853f12'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:23.945605 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] The token cache is being invalidate because project a70d5a50b851465d88ed85969f9d37d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:34:23.945915 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:23.946569 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:23.948246 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1439/2868] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:34:23 2022] DELETE /identity/v3/projects/a70d5a50b851465d88ed85969f9d37d3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:23.952655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de5dc8-60b8-4cee-9f2d-5dc4b99192bd service neutron] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:23.962801 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de5dc8-60b8-4cee-9f2d-5dc4b99192bd service neutron] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:24.021528 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:24.022260 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:24.022598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:24.022884 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] PATH_INFO: `/v3/users/a2e2d2bf899e41db9d725c6fea70e768` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:24.024005 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e2d2bf899e41db9d725c6fea70e768)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:24.025914 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:24.238129 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e2d2bf899e41db9d725c6fea70e768'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:24.332622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e2d2bf899e41db9d725c6fea70e768'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:24.333224 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e2d2bf899e41db9d725c6fea70e768'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:24.341051 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e2d2bf899e41db9d725c6fea70e768'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:24.353608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e2d2bf899e41db9d725c6fea70e768'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:24.360489 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1430/2869] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:34:23 2022] DELETE /identity/v3/users/a2e2d2bf899e41db9d725c6fea70e768 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 09 23:34:24.779290 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:24.780902 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:24.800362 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b160e-2bd6-443f-849d-2f0857853f12 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:24.849042 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:24.858944 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:24.860499 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:24.860734 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:24.860875 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] PATH_INFO: `/v3/projects/b0246d177bcf409f8ee630e80e72e967` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:24.862029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0246d177bcf409f8ee630e80e72e967)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:24.863973 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:24.967291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0246d177bcf409f8ee630e80e72e967', 'request_id': 'req-3429a443-0398-451d-bb7b-bb2bab27f3fd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.022967 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0246d177bcf409f8ee630e80e72e967', 'request_id': 'req-3429a443-0398-451d-bb7b-bb2bab27f3fd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.028710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0246d177bcf409f8ee630e80e72e967', 'request_id': 'req-3429a443-0398-451d-bb7b-bb2bab27f3fd'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.040476 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] The token cache is being invalidate because project b0246d177bcf409f8ee630e80e72e967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:34:25.040696 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.041324 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.042848 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1440/2870] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:34:24 2022] DELETE /identity/v3/projects/b0246d177bcf409f8ee630e80e72e967 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:25.047450 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:25.048930 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:25.062025 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51219b80-954b-4e04-bc58-ac6bb4c5f7f8 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:25.119042 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:25.119671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:25.119952 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:25.120229 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] PATH_INFO: `/v3/users/e7c581ddfb954607aea27232f26f4523` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:25.121343 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c581ddfb954607aea27232f26f4523)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:25.123302 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:25.128659 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:25.311549 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c581ddfb954607aea27232f26f4523'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.366676 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c581ddfb954607aea27232f26f4523'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.367306 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c581ddfb954607aea27232f26f4523'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.374662 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c581ddfb954607aea27232f26f4523'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.386715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-63ffb8ed-0431-4fd6-8ee4-81b9801802d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c581ddfb954607aea27232f26f4523'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:25.393328 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1431/2871] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:34:25 2022] DELETE /identity/v3/users/e7c581ddfb954607aea27232f26f4523 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:25.762573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:25.763972 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:25.777571 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429a443-0398-451d-bb7b-bb2bab27f3fd admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:25.837007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:25.837563 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:25.837840 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:25.838110 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] PATH_INFO: `/v3/projects/8d50ed1146bf4c499fafefbe6ff3de05` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:25.839253 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d50ed1146bf4c499fafefbe6ff3de05)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:25.841162 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:25.939464 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d50ed1146bf4c499fafefbe6ff3de05', 'request_id': 'req-37958c4c-4c39-473b-b34e-b6b02a8ef13d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:26.011719 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d50ed1146bf4c499fafefbe6ff3de05', 'request_id': 'req-37958c4c-4c39-473b-b34e-b6b02a8ef13d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:26.018012 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d50ed1146bf4c499fafefbe6ff3de05', 'request_id': 'req-37958c4c-4c39-473b-b34e-b6b02a8ef13d'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:26.029234 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] The token cache is being invalidate because project 8d50ed1146bf4c499fafefbe6ff3de05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81770) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:34:26.029522 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:26.030154 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-37958c4c-4c39-473b-b34e-b6b02a8ef13d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:26.031821 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1441/2872] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:34:25 2022] DELETE /identity/v3/projects/8d50ed1146bf4c499fafefbe6ff3de05 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:26.037740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b207af1-4ffc-4f4a-9095-16cb41234937 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.038026 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b207af1-4ffc-4f4a-9095-16cb41234937 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.038303 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b207af1-4ffc-4f4a-9095-16cb41234937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.057999 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-6b207af1-4ffc-4f4a-9095-16cb41234937 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:26.059238 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-6b207af1-4ffc-4f4a-9095-16cb41234937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:26.118571 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1432/2873] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:26.127595 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523c1a-80d6-4274-ab12-2896fe89231c None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.127883 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523c1a-80d6-4274-ab12-2896fe89231c None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.128188 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa523c1a-80d6-4274-ab12-2896fe89231c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.147862 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-fa523c1a-80d6-4274-ab12-2896fe89231c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:26.149065 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-fa523c1a-80d6-4274-ab12-2896fe89231c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:26.156864 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1442/2874] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:26.161067 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b207af1-4ffc-4f4a-9095-16cb41234937 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:26.170598 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b207af1-4ffc-4f4a-9095-16cb41234937 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:26.175897 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:26.176479 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.176751 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.177018 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.177671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81769) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:34:26.177961 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:26.179567 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:26.191667 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1433/2875] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:34:26.195897 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523c1a-80d6-4274-ab12-2896fe89231c None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:26.201454 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:26.205155 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa523c1a-80d6-4274-ab12-2896fe89231c None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:26.209904 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832d6d-fb76-4484-b0ae-cc2a1938cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:26.214580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832d6d-fb76-4484-b0ae-cc2a1938cbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.214580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832d6d-fb76-4484-b0ae-cc2a1938cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.214580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30832d6d-fb76-4484-b0ae-cc2a1938cbf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.214580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832d6d-fb76-4484-b0ae-cc2a1938cbf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:26.214580 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30832d6d-fb76-4484-b0ae-cc2a1938cbf6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:26.276119 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1443/2876] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:34:26 2022] POST /identity/v3/projects => generated 387 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:34:26.280758 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:26.290906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6747e4e-d12b-4445-a0e5-adcc349f71e1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:26.347248 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2c225-9c66-4537-87bf-6676fcb4608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:26.347848 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2c225-9c66-4537-87bf-6676fcb4608f admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.348124 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2c225-9c66-4537-87bf-6676fcb4608f admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.348394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2c225-9c66-4537-87bf-6676fcb4608f admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.349011 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2c225-9c66-4537-87bf-6676fcb4608f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:26.350967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2c225-9c66-4537-87bf-6676fcb4608f admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:26.441753 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1434/2877] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:34:26 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:26.445847 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832d6d-fb76-4484-b0ae-cc2a1938cbf6 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:26.456823 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30832d6d-fb76-4484-b0ae-cc2a1938cbf6 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:26.462060 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6877046-e123-481e-9dc9-92e72160e9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:26.462685 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6877046-e123-481e-9dc9-92e72160e9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.462962 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6877046-e123-481e-9dc9-92e72160e9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.463230 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6877046-e123-481e-9dc9-92e72160e9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.463874 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6877046-e123-481e-9dc9-92e72160e9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:26.465455 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6877046-e123-481e-9dc9-92e72160e9e7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:26.477203 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1444/2878] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:26.482615 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2c225-9c66-4537-87bf-6676fcb4608f admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:26.489070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:26.494645 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2c225-9c66-4537-87bf-6676fcb4608f admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:26.499369 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd6951-0359-4358-8908-43a739bcd510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:26.500487 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6951-0359-4358-8908-43a739bcd510 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.500777 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6951-0359-4358-8908-43a739bcd510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.501052 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd6951-0359-4358-8908-43a739bcd510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.501684 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd6951-0359-4358-8908-43a739bcd510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:26.503423 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd6951-0359-4358-8908-43a739bcd510 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:26.514397 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1435/2879] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:26.519049 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6877046-e123-481e-9dc9-92e72160e9e7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:26.528980 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6877046-e123-481e-9dc9-92e72160e9e7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:26.534622 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984b14a-64b8-4f00-8358-ab4d1f3837e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:26.535273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b14a-64b8-4f00-8358-ab4d1f3837e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.535552 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b14a-64b8-4f00-8358-ab4d1f3837e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.535824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984b14a-64b8-4f00-8358-ab4d1f3837e9 admin admin] PATH_INFO: `/v3/projects/a3f86f6b5774403181d7a9b2a4b2a220/users/5a3c54ad38994110a468b11beb178a3c/roles/453242bdc829422490366038c62c149a` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.551633 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984b14a-64b8-4f00-8358-ab4d1f3837e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f86f6b5774403181d7a9b2a4b2a220, user_id=5a3c54ad38994110a468b11beb178a3c, role_id=453242bdc829422490366038c62c149a)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:26.554359 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984b14a-64b8-4f00-8358-ab4d1f3837e9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:26.612021 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1445/2880] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:26 2022] PUT /identity/v3/projects/a3f86f6b5774403181d7a9b2a4b2a220/users/5a3c54ad38994110a468b11beb178a3c/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:26.624946 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7386e3-46e9-4071-a25f-24a4ef1fccb5 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.625226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7386e3-46e9-4071-a25f-24a4ef1fccb5 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.625494 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7386e3-46e9-4071-a25f-24a4ef1fccb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.652112 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-9a7386e3-46e9-4071-a25f-24a4ef1fccb5 None None] MFA Rules not processed for user `5a3c54ad38994110a468b11beb178a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:26.653299 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-9a7386e3-46e9-4071-a25f-24a4ef1fccb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:26.715938 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1436/2881] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:26.721029 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984b14a-64b8-4f00-8358-ab4d1f3837e9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:26.784556 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984b14a-64b8-4f00-8358-ab4d1f3837e9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:26.789273 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce828e4-65ae-442a-8525-8d5f55bfe653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:26.789869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce828e4-65ae-442a-8525-8d5f55bfe653 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.790146 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce828e4-65ae-442a-8525-8d5f55bfe653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.790448 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce828e4-65ae-442a-8525-8d5f55bfe653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.791079 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce828e4-65ae-442a-8525-8d5f55bfe653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:26.792955 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce828e4-65ae-442a-8525-8d5f55bfe653 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:26.853325 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1446/2882] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:34:26 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:26.857740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7386e3-46e9-4071-a25f-24a4ef1fccb5 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:26.869046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7386e3-46e9-4071-a25f-24a4ef1fccb5 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:26.924262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a85a8-54f9-4d60-a1fe-b16151c32479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:26.924875 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a85a8-54f9-4d60-a1fe-b16151c32479 admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:26.925160 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a85a8-54f9-4d60-a1fe-b16151c32479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:26.925435 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a85a8-54f9-4d60-a1fe-b16151c32479 admin admin] PATH_INFO: `/v3/users` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:26.926076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a85a8-54f9-4d60-a1fe-b16151c32479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:26.929029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a85a8-54f9-4d60-a1fe-b16151c32479 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:27.012364 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1437/2883] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:34:26 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:27.016771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce828e4-65ae-442a-8525-8d5f55bfe653 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:27.027214 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce828e4-65ae-442a-8525-8d5f55bfe653 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:27.034041 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d9d61-0e72-48d9-a268-0a574f6290b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:27.034673 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d9d61-0e72-48d9-a268-0a574f6290b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:27.034953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d9d61-0e72-48d9-a268-0a574f6290b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:27.035229 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d9d61-0e72-48d9-a268-0a574f6290b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:27.035866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d9d61-0e72-48d9-a268-0a574f6290b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:27.037440 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d9d61-0e72-48d9-a268-0a574f6290b3 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:27.048517 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1447/2884] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:27.052582 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a85a8-54f9-4d60-a1fe-b16151c32479 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:27.063338 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a85a8-54f9-4d60-a1fe-b16151c32479 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:27.068454 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51359a70-1d47-4793-acc9-b21ec48ba4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:27.069189 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51359a70-1d47-4793-acc9-b21ec48ba4cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:27.069504 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51359a70-1d47-4793-acc9-b21ec48ba4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:27.069822 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51359a70-1d47-4793-acc9-b21ec48ba4cd admin admin] PATH_INFO: `/v3/projects/3f6b75a86b9d40dc8dbab472e50e5393/users/e3450c914fd34b5b83c7d5032ff01b1b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:27.086013 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51359a70-1d47-4793-acc9-b21ec48ba4cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6b75a86b9d40dc8dbab472e50e5393, user_id=e3450c914fd34b5b83c7d5032ff01b1b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:27.088762 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51359a70-1d47-4793-acc9-b21ec48ba4cd admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:27.161142 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1438/2885] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:27 2022] PUT /identity/v3/projects/3f6b75a86b9d40dc8dbab472e50e5393/users/e3450c914fd34b5b83c7d5032ff01b1b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:27.165082 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d9d61-0e72-48d9-a268-0a574f6290b3 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:27.175559 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d9d61-0e72-48d9-a268-0a574f6290b3 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:27.231942 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d0806-ba06-48b7-9679-d74ed84924f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:27.232538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0806-ba06-48b7-9679-d74ed84924f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:27.232814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0806-ba06-48b7-9679-d74ed84924f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:27.233085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d0806-ba06-48b7-9679-d74ed84924f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:27.233713 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d0806-ba06-48b7-9679-d74ed84924f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:27.235753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d0806-ba06-48b7-9679-d74ed84924f2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:27.245660 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1448/2886] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:27.250417 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51359a70-1d47-4793-acc9-b21ec48ba4cd admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:27.260800 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51359a70-1d47-4793-acc9-b21ec48ba4cd admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:27.266556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:27.266707 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:27.266982 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:27.267254 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] PATH_INFO: `/v3/projects/3f6b75a86b9d40dc8dbab472e50e5393/users/e3450c914fd34b5b83c7d5032ff01b1b/roles/c935d5f3d40f451d9e734a3fd8077e22` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:27.269170 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6b75a86b9d40dc8dbab472e50e5393, user_id=e3450c914fd34b5b83c7d5032ff01b1b, role_id=c935d5f3d40f451d9e734a3fd8077e22)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:27.271824 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:27.281995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1439/2887] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:27 2022] PUT /identity/v3/projects/3f6b75a86b9d40dc8dbab472e50e5393/users/e3450c914fd34b5b83c7d5032ff01b1b/roles/c935d5f3d40f451d9e734a3fd8077e22 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:27.291818 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced61bd-3a96-419b-ad8d-645107c4ee56 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:27.292101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced61bd-3a96-419b-ad8d-645107c4ee56 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:27.292372 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced61bd-3a96-419b-ad8d-645107c4ee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:27.319845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-6ced61bd-3a96-419b-ad8d-645107c4ee56 None None] MFA Rules not processed for user `e3450c914fd34b5b83c7d5032ff01b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:27.321034 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-6ced61bd-3a96-419b-ad8d-645107c4ee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:27.325845 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:27.388173 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1449/2888] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:27.401322 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:27.401561 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:27.438626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:27.461404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7201a-78fa-4f2d-a87f-4db094348ae4 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:27.465324 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920c595-a7b9-4e91-ad3e-87728ac6125a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:27.465941 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c595-a7b9-4e91-ad3e-87728ac6125a service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:27.466221 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c595-a7b9-4e91-ad3e-87728ac6125a service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:27.466531 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c595-a7b9-4e91-ad3e-87728ac6125a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:27.476985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920c595-a7b9-4e91-ad3e-87728ac6125a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:27.478460 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920c595-a7b9-4e91-ad3e-87728ac6125a service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:27.491671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1440/2889] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:27.628581 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced61bd-3a96-419b-ad8d-645107c4ee56 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:27.689176 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced61bd-3a96-419b-ad8d-645107c4ee56 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:27.693676 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:27.694294 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:27.694611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:27.694886 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] PATH_INFO: `/v3/users/5a3c54ad38994110a468b11beb178a3c` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:27.695987 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a3c54ad38994110a468b11beb178a3c)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:27.697924 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:27.860071 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a3c54ad38994110a468b11beb178a3c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:27.860832 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a3c54ad38994110a468b11beb178a3c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:27.914401 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a3c54ad38994110a468b11beb178a3c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:27.927083 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a3c54ad38994110a468b11beb178a3c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:27.934223 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a3c54ad38994110a468b11beb178a3c'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:27.941170 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1450/2890] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:34:27 2022] DELETE /identity/v3/users/5a3c54ad38994110a468b11beb178a3c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:27.945688 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e247ff-7670-4e6d-bc9b-2e2f31eabd07 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:27.945967 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e247ff-7670-4e6d-bc9b-2e2f31eabd07 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:27.946233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e247ff-7670-4e6d-bc9b-2e2f31eabd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:27.965987 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-c0e247ff-7670-4e6d-bc9b-2e2f31eabd07 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:27.967206 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c0e247ff-7670-4e6d-bc9b-2e2f31eabd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:28.028149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1441/2891] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:28.037779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:28.039202 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:28.066652 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:28.097603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ff7da-d0ed-4f15-b284-1467c6631786 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:28.101763 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f57f41-737f-4299-8269-5d9016d76f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:28.102396 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f57f41-737f-4299-8269-5d9016d76f30 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:28.102672 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f57f41-737f-4299-8269-5d9016d76f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:28.102941 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f57f41-737f-4299-8269-5d9016d76f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:28.112599 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f57f41-737f-4299-8269-5d9016d76f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:28.113953 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f57f41-737f-4299-8269-5d9016d76f30 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:28.123866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1451/2892] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:28.517399 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e247ff-7670-4e6d-bc9b-2e2f31eabd07 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:28.520799 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-c0e247ff-7670-4e6d-bc9b-2e2f31eabd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:28.531951 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e247ff-7670-4e6d-bc9b-2e2f31eabd07 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:28.536655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:28.537273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:28.537564 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:28.537823 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] PATH_INFO: `/v3/projects/a3f86f6b5774403181d7a9b2a4b2a220` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:28.539163 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f86f6b5774403181d7a9b2a4b2a220)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:28.541294 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:28.646954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f86f6b5774403181d7a9b2a4b2a220', 'request_id': 'req-464b9e1c-70c6-4689-b78c-49d749ff962c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.697394 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f86f6b5774403181d7a9b2a4b2a220', 'request_id': 'req-464b9e1c-70c6-4689-b78c-49d749ff962c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.703630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f86f6b5774403181d7a9b2a4b2a220', 'request_id': 'req-464b9e1c-70c6-4689-b78c-49d749ff962c'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.714842 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] The token cache is being invalidate because project a3f86f6b5774403181d7a9b2a4b2a220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:34:28.715123 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.715788 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.717300 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1442/2893] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:34:28 2022] DELETE /identity/v3/projects/a3f86f6b5774403181d7a9b2a4b2a220 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:28.722498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f57f41-737f-4299-8269-5d9016d76f30 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:28.731966 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f57f41-737f-4299-8269-5d9016d76f30 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:28.789061 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:28.789677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:28.789960 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:28.790215 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] PATH_INFO: `/v3/users/e3450c914fd34b5b83c7d5032ff01b1b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:28.791347 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3450c914fd34b5b83c7d5032ff01b1b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:28.793240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:28.917052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3450c914fd34b5b83c7d5032ff01b1b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.917774 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3450c914fd34b5b83c7d5032ff01b1b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.960607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3450c914fd34b5b83c7d5032ff01b1b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.973484 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3450c914fd34b5b83c7d5032ff01b1b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.980213 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-67d2ef30-d138-402b-8415-20de594f26a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3450c914fd34b5b83c7d5032ff01b1b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:28.986153 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1452/2894] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:34:28 2022] DELETE /identity/v3/users/e3450c914fd34b5b83c7d5032ff01b1b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:29.402090 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:29.403648 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:29.425505 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b9e1c-70c6-4689-b78c-49d749ff962c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:29.488132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:29.488681 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:29.488955 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:29.489226 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] PATH_INFO: `/v3/projects/3f6b75a86b9d40dc8dbab472e50e5393` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:29.490619 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f6b75a86b9d40dc8dbab472e50e5393)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:29.492525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:29.493956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:29.595937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f6b75a86b9d40dc8dbab472e50e5393', 'request_id': 'req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:29.669096 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f6b75a86b9d40dc8dbab472e50e5393', 'request_id': 'req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:29.675530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f6b75a86b9d40dc8dbab472e50e5393', 'request_id': 'req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:29.688075 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] The token cache is being invalidate because project 3f6b75a86b9d40dc8dbab472e50e5393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:34:29.688539 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:29.689174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bfb4f2e-2d02-4141-adc6-5f6ee57f6df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:29.690689 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1443/2895] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:34:29 2022] DELETE /identity/v3/projects/3f6b75a86b9d40dc8dbab472e50e5393 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:29.696759 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38975b67-e4c6-4bff-af08-3bbca6bd0336 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:29.697227 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38975b67-e4c6-4bff-af08-3bbca6bd0336 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:29.697329 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38975b67-e4c6-4bff-af08-3bbca6bd0336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:29.716677 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-38975b67-e4c6-4bff-af08-3bbca6bd0336 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:29.717858 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-38975b67-e4c6-4bff-af08-3bbca6bd0336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:29.758642 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:29.776718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1453/2896] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:29.786050 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740f340-2f62-4e02-92a0-7950b4ef8b44 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:29.786367 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740f340-2f62-4e02-92a0-7950b4ef8b44 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:29.786655 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740f340-2f62-4e02-92a0-7950b4ef8b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:29.807586 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-7740f340-2f62-4e02-92a0-7950b4ef8b44 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:29.808797 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-7740f340-2f62-4e02-92a0-7950b4ef8b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:29.817734 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1444/2897] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:29.822183 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38975b67-e4c6-4bff-af08-3bbca6bd0336 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:29.832380 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38975b67-e4c6-4bff-af08-3bbca6bd0336 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:29.836893 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:29.837498 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:29.837771 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:29.838035 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:29.838701 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:34:29.838996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:29.840669 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:29.852333 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1454/2898] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:34:29.856530 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740f340-2f62-4e02-92a0-7950b4ef8b44 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:29.866907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740f340-2f62-4e02-92a0-7950b4ef8b44 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:29.871546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde8ce6-9972-4b81-9f0e-84784cc6d44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:29.872132 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8ce6-9972-4b81-9f0e-84784cc6d44a admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:29.872404 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8ce6-9972-4b81-9f0e-84784cc6d44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:29.872671 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8ce6-9972-4b81-9f0e-84784cc6d44a admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:29.873297 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde8ce6-9972-4b81-9f0e-84784cc6d44a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:29.876255 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde8ce6-9972-4b81-9f0e-84784cc6d44a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:29.956107 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1445/2899] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:34:29 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:34:29.960142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:29.970866 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c238e-ea1c-4c4b-96a3-07e5260c4eb9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:30.030590 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a4bc6-c59f-4296-a5e9-d5415de2b8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:30.032233 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a4bc6-c59f-4296-a5e9-d5415de2b8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:30.032523 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a4bc6-c59f-4296-a5e9-d5415de2b8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:30.032796 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a4bc6-c59f-4296-a5e9-d5415de2b8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:30.033421 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a4bc6-c59f-4296-a5e9-d5415de2b8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:30.035350 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a4bc6-c59f-4296-a5e9-d5415de2b8e2 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:30.137843 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1455/2900] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:34:29 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:30.142760 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde8ce6-9972-4b81-9f0e-84784cc6d44a admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:30.155242 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde8ce6-9972-4b81-9f0e-84784cc6d44a admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:30.162262 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e025288-4ef6-4117-bf9b-12292244d9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:30.162956 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e025288-4ef6-4117-bf9b-12292244d9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:30.163234 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e025288-4ef6-4117-bf9b-12292244d9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:30.163525 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e025288-4ef6-4117-bf9b-12292244d9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:30.164165 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e025288-4ef6-4117-bf9b-12292244d9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:30.165768 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e025288-4ef6-4117-bf9b-12292244d9a1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:30.175996 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1446/2901] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:30.180057 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a4bc6-c59f-4296-a5e9-d5415de2b8e2 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:30.190814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a4bc6-c59f-4296-a5e9-d5415de2b8e2 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:30.195460 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f7125-b81b-4173-b7f5-048eae609711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:30.196048 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f7125-b81b-4173-b7f5-048eae609711 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:30.196321 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f7125-b81b-4173-b7f5-048eae609711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:30.196585 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f7125-b81b-4173-b7f5-048eae609711 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:30.197208 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f7125-b81b-4173-b7f5-048eae609711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:30.198814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123f7125-b81b-4173-b7f5-048eae609711 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:30.210050 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1456/2902] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:30.214094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e025288-4ef6-4117-bf9b-12292244d9a1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:30.224246 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e025288-4ef6-4117-bf9b-12292244d9a1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:30.228839 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:30.229477 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:30.229756 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:30.230031 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] PATH_INFO: `/v3/projects/4e15a48713c14b40aef62e1ea94fb88f/users/50d16a1ff11c40cc9bc14f05c01b3c31/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:30.244937 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e15a48713c14b40aef62e1ea94fb88f, user_id=50d16a1ff11c40cc9bc14f05c01b3c31, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:30.247626 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:30.303620 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1447/2903] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:30 2022] PUT /identity/v3/projects/4e15a48713c14b40aef62e1ea94fb88f/users/50d16a1ff11c40cc9bc14f05c01b3c31/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:30.314009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46ef8-8722-40c8-996e-422b8a2b1057 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:30.314291 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46ef8-8722-40c8-996e-422b8a2b1057 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:30.314592 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46ef8-8722-40c8-996e-422b8a2b1057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:30.342603 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-06e46ef8-8722-40c8-996e-422b8a2b1057 None None] MFA Rules not processed for user `50d16a1ff11c40cc9bc14f05c01b3c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:30.344026 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-06e46ef8-8722-40c8-996e-422b8a2b1057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:30.408085 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1457/2904] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:30.419622 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:30.421340 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:30.477832 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7fff2-522e-4b28-97b3-c3eb696d478c admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:30.481545 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac7933-a3d4-403d-bd85-d365d18ba7d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:30.482149 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac7933-a3d4-403d-bd85-d365d18ba7d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:30.482463 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac7933-a3d4-403d-bd85-d365d18ba7d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:30.482740 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac7933-a3d4-403d-bd85-d365d18ba7d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:30.492355 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac7933-a3d4-403d-bd85-d365d18ba7d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:30.493861 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac7933-a3d4-403d-bd85-d365d18ba7d0 service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:30.504277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1448/2905] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:40.133543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46ef8-8722-40c8-996e-422b8a2b1057 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:40.174624 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:40.194308 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46ef8-8722-40c8-996e-422b8a2b1057 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:40.198822 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:40.199468 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:40.199948 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:40.200433 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] PATH_INFO: `/v3/users/50d16a1ff11c40cc9bc14f05c01b3c31` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:40.201992 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d16a1ff11c40cc9bc14f05c01b3c31)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:40.203946 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:40.339783 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d16a1ff11c40cc9bc14f05c01b3c31'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:40.340456 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d16a1ff11c40cc9bc14f05c01b3c31'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:40.388544 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d16a1ff11c40cc9bc14f05c01b3c31'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:40.401449 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d16a1ff11c40cc9bc14f05c01b3c31'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:40.409414 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d16a1ff11c40cc9bc14f05c01b3c31'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:40.418892 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1458/2906] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:34:40 2022] DELETE /identity/v3/users/50d16a1ff11c40cc9bc14f05c01b3c31 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:40.423954 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0eafc-e863-4674-a991-23586d46c3c4 None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:40.424466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0eafc-e863-4674-a991-23586d46c3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:40.424466 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0eafc-e863-4674-a991-23586d46c3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:40.443372 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-03d0eafc-e863-4674-a991-23586d46c3c4 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:40.444579 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-03d0eafc-e863-4674-a991-23586d46c3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:40.467950 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:40.504898 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1449/2907] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:40.514634 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:40.516804 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:40.574546 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3323fe8-7dd2-4f00-b500-7df690b0416c admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:40.578190 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73cdf9-18e8-4f5d-b198-401c71917f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:40.578817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73cdf9-18e8-4f5d-b198-401c71917f57 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:40.579093 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73cdf9-18e8-4f5d-b198-401c71917f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:40.579361 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73cdf9-18e8-4f5d-b198-401c71917f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:40.588196 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73cdf9-18e8-4f5d-b198-401c71917f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:40.589611 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73cdf9-18e8-4f5d-b198-401c71917f57 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:40.599307 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1459/2908] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:34:41.035070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0eafc-e863-4674-a991-23586d46c3c4 None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:41.039076 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-03d0eafc-e863-4674-a991-23586d46c3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:41.049207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0eafc-e863-4674-a991-23586d46c3c4 None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:41.055245 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:41.055901 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.056176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.056532 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] PATH_INFO: `/v3/projects/4e15a48713c14b40aef62e1ea94fb88f` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.057650 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e15a48713c14b40aef62e1ea94fb88f)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:41.059575 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:41.183985 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e15a48713c14b40aef62e1ea94fb88f', 'request_id': 'req-12cbf488-7e48-46f4-8175-0ef24145b45a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:41.257670 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e15a48713c14b40aef62e1ea94fb88f', 'request_id': 'req-12cbf488-7e48-46f4-8175-0ef24145b45a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:41.263498 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e15a48713c14b40aef62e1ea94fb88f', 'request_id': 'req-12cbf488-7e48-46f4-8175-0ef24145b45a'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:41.275094 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] The token cache is being invalidate because project 4e15a48713c14b40aef62e1ea94fb88f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:34:41.275321 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:41.275929 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-12cbf488-7e48-46f4-8175-0ef24145b45a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:34:41.277414 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1450/2909] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:34:41 2022] DELETE /identity/v3/projects/4e15a48713c14b40aef62e1ea94fb88f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:41.282865 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e259-ce4c-4ca0-8566-dc83a39f9297 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.283373 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e259-ce4c-4ca0-8566-dc83a39f9297 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.283491 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9e259-ce4c-4ca0-8566-dc83a39f9297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.303586 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-acd9e259-ce4c-4ca0-8566-dc83a39f9297 None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:41.304777 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-acd9e259-ce4c-4ca0-8566-dc83a39f9297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:41.364095 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1460/2910] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:41.371990 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406aef1-f6ed-4beb-b213-6f319df5262c None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.372272 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406aef1-f6ed-4beb-b213-6f319df5262c None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.372546 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406aef1-f6ed-4beb-b213-6f319df5262c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.393701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-3406aef1-f6ed-4beb-b213-6f319df5262c None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:41.395014 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-3406aef1-f6ed-4beb-b213-6f319df5262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:41.402880 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1451/2911] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:41.407710 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9e259-ce4c-4ca0-8566-dc83a39f9297 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:41.412614 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:41.417824 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9e259-ce4c-4ca0-8566-dc83a39f9297 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:41.422975 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:41.423597 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.423869 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.424135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.425007 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81770) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 09 23:34:41.425181 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:41.426817 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:41.438534 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1461/2912] 10.222.0.20 () {58 vars in 1153 bytes} [Fri Sep 9 23:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 09 23:34:41.442907 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406aef1-f6ed-4beb-b213-6f319df5262c None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:41.452110 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406aef1-f6ed-4beb-b213-6f319df5262c None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:41.458003 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb1e4a-7854-48f4-a02b-69de44a764ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:41.458629 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb1e4a-7854-48f4-a02b-69de44a764ef admin admin] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.458906 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb1e4a-7854-48f4-a02b-69de44a764ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.459174 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb1e4a-7854-48f4-a02b-69de44a764ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.459808 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb1e4a-7854-48f4-a02b-69de44a764ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:41.461701 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb1e4a-7854-48f4-a02b-69de44a764ef admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:41.556070 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1452/2913] 10.222.0.20 () {60 vars in 1153 bytes} [Fri Sep 9 23:34:41 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 09 23:34:41.560175 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:41.571128 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfbb9-e410-4293-ad92-02b7a256dfc7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:41.628879 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f0a91-06f2-4b60-b1d6-f98058ae21d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:41.629477 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f0a91-06f2-4b60-b1d6-f98058ae21d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.629752 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f0a91-06f2-4b60-b1d6-f98058ae21d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.630020 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f0a91-06f2-4b60-b1d6-f98058ae21d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.630682 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f0a91-06f2-4b60-b1d6-f98058ae21d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:41.632570 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f0a91-06f2-4b60-b1d6-f98058ae21d7 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:41.697583 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1462/2914] 10.222.0.20 () {60 vars in 1144 bytes} [Fri Sep 9 23:34:41 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 09 23:34:41.701870 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb1e4a-7854-48f4-a02b-69de44a764ef admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:41.712602 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb1e4a-7854-48f4-a02b-69de44a764ef admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:41.717207 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b826f8d-89a5-4de4-b7dd-66dd8ca396b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:41.717787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b826f8d-89a5-4de4-b7dd-66dd8ca396b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.718064 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b826f8d-89a5-4de4-b7dd-66dd8ca396b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.718317 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b826f8d-89a5-4de4-b7dd-66dd8ca396b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.720021 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b826f8d-89a5-4de4-b7dd-66dd8ca396b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:41.721614 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b826f8d-89a5-4de4-b7dd-66dd8ca396b1 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:41.731391 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1453/2915] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:41.736024 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f0a91-06f2-4b60-b1d6-f98058ae21d7 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:41.746204 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f0a91-06f2-4b60-b1d6-f98058ae21d7 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:41.751101 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284f8f7-38d5-4316-b707-90c9d2076e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:41.751688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f8f7-38d5-4316-b707-90c9d2076e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.751963 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f8f7-38d5-4316-b707-90c9d2076e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.752252 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284f8f7-38d5-4316-b707-90c9d2076e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.752880 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284f8f7-38d5-4316-b707-90c9d2076e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:41.754538 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284f8f7-38d5-4316-b707-90c9d2076e29 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:41.765100 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1463/2916] 10.222.0.20 () {58 vars in 1122 bytes} [Fri Sep 9 23:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 09 23:34:41.769161 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b826f8d-89a5-4de4-b7dd-66dd8ca396b1 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:41.779084 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b826f8d-89a5-4de4-b7dd-66dd8ca396b1 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:41.784634 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:41.785273 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] REQUEST_METHOD: `PUT` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.797304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.797304 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] PATH_INFO: `/v3/projects/4a5e61a90a1043e7a03fbfb12e317966/users/720314c72fc449c1a60c8bcd96fd5604/roles/453242bdc829422490366038c62c149a` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.800702 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5e61a90a1043e7a03fbfb12e317966, user_id=720314c72fc449c1a60c8bcd96fd5604, role_id=453242bdc829422490366038c62c149a)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:41.803474 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:41.868885 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1454/2917] 10.222.0.20 () {60 vars in 1483 bytes} [Fri Sep 9 23:34:41 2022] PUT /identity/v3/projects/4a5e61a90a1043e7a03fbfb12e317966/users/720314c72fc449c1a60c8bcd96fd5604/roles/453242bdc829422490366038c62c149a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:34:41.881111 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55fbfa-7d56-4466-987e-b846674ebe24 None None] REQUEST_METHOD: `POST` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:41.881394 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55fbfa-7d56-4466-987e-b846674ebe24 None None] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:41.881682 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55fbfa-7d56-4466-987e-b846674ebe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:41.909684 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.auth.core [None req-4f55fbfa-7d56-4466-987e-b846674ebe24 None None] MFA Rules not processed for user `720314c72fc449c1a60c8bcd96fd5604`. Rule list: `[]` (Enabled: `True`). {{(pid=81770) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:34:41.910929 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4f55fbfa-7d56-4466-987e-b846674ebe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:41.976607 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1464/2918] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:34:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:34:41.987856 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:34:41.989497 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:34:42.041724 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:42.045370 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20738f09-304f-430f-8e4a-90692d7bc564 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:34:42.049918 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314dfde-e754-4f11-978d-f1a279f2a50f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c54fab93990146e0bb746abfd0843444', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:34:42.050541 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314dfde-e754-4f11-978d-f1a279f2a50f service cinder] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:34:42.055252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314dfde-e754-4f11-978d-f1a279f2a50f service cinder] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:34:42.055252 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314dfde-e754-4f11-978d-f1a279f2a50f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:34:42.059192 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314dfde-e754-4f11-978d-f1a279f2a50f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:34:42.060608 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314dfde-e754-4f11-978d-f1a279f2a50f service cinder] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:34:42.070152 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1455/2919] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:34:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:34:51.973807 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:34:52.126765 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=4 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:35:05.061216 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55fbfa-7d56-4466-987e-b846674ebe24 None None] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:35:05.123046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55fbfa-7d56-4466-987e-b846674ebe24 None None] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:35:05.127526 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:35:05.128152 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:05.128429 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:05.128826 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] PATH_INFO: `/v3/users/720314c72fc449c1a60c8bcd96fd5604` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:05.130056 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=720314c72fc449c1a60c8bcd96fd5604)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:35:05.131997 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:35:05.311610 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720314c72fc449c1a60c8bcd96fd5604'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:05.312231 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720314c72fc449c1a60c8bcd96fd5604'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:05.359891 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720314c72fc449c1a60c8bcd96fd5604'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:05.372784 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720314c72fc449c1a60c8bcd96fd5604'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:05.380039 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720314c72fc449c1a60c8bcd96fd5604'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:05.386076 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1465/2920] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:35:05 2022] DELETE /identity/v3/users/720314c72fc449c1a60c8bcd96fd5604 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:35:05.390642 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2f277-d0d0-4827-b814-328201a151af None None] REQUEST_METHOD: `POST` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:05.391157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2f277-d0d0-4827-b814-328201a151af None None] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:05.391157 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2f277-d0d0-4827-b814-328201a151af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:05.411197 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.auth.core [None req-2df2f277-d0d0-4827-b814-328201a151af None None] MFA Rules not processed for user `e84ee63259eb48d0a32d4ab69da0f378`. Rule list: `[]` (Enabled: `True`). {{(pid=81769) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 09 23:35:05.412438 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2df2f277-d0d0-4827-b814-328201a151af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:05.469928 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1456/2921] 10.222.0.20 () {58 vars in 958 bytes} [Fri Sep 9 23:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 09 23:35:05.480358 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:35:05.481800 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:05.537814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7398-a93e-494f-a552-87fcaad398e9 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:35:05.541403 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:35:05.541991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:05.542260 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:05.542620 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:05.551445 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:35:05.552838 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:35:05.562066 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1466/2922] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:35:05.945709 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2f277-d0d0-4827-b814-328201a151af None None] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:35:05.947379 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-2df2f277-d0d0-4827-b814-328201a151af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:05.960611 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2f277-d0d0-4827-b814-328201a151af None None] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:35:05.970215 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:35:05.970892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:05.974803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:05.974803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] PATH_INFO: `/v3/projects/4a5e61a90a1043e7a03fbfb12e317966` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:05.974803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5e61a90a1043e7a03fbfb12e317966)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:35:05.974803 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:35:06.103175 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5e61a90a1043e7a03fbfb12e317966', 'request_id': 'req-37723047-7016-4dbe-92e1-d7041edd4161'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:06.177838 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5e61a90a1043e7a03fbfb12e317966', 'request_id': 'req-37723047-7016-4dbe-92e1-d7041edd4161'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:06.183892 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5e61a90a1043e7a03fbfb12e317966', 'request_id': 'req-37723047-7016-4dbe-92e1-d7041edd4161'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:06.195966 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] The token cache is being invalidate because project 4a5e61a90a1043e7a03fbfb12e317966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:35:06.196792 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:06.196995 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:06.198544 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1457/2923] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:35:05 2022] DELETE /identity/v3/projects/4a5e61a90a1043e7a03fbfb12e317966 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:35:15.557922 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:35:16.192973 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:35:51.428014 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:35:51.429409 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:51.440814 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d61780-9fe2-4a79-8829-428b4868bc68 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:35:51.490363 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9da2d7ef80f4622b58c9041b65df992', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:35:51.490991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:51.491271 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:51.491543 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:51.493582 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:51.556407 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:35:51.557825 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:35:51.567408 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1467/2924] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:35:55.090682 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:35:55.092176 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:55.172794 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37723047-7016-4dbe-92e1-d7041edd4161 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:35:55.177869 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:35:55.178511 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] REQUEST_METHOD: `GET` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:55.178795 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:55.179111 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:55.180940 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:55.192836 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:35:55.194455 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:35:55.206114 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1458/2925] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:35:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 09 23:35:58.823996 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:35:58.825545 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:58.838240 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f55a5-de86-445d-bdba-62eae83378a0 service neutron] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:35:58.843142 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:35:58.843779 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:58.844046 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:58.844315 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] PATH_INFO: `/v3/users/8c81c0af6f084b8db165fd51667b3b54` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:58.845646 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c81c0af6f084b8db165fd51667b3b54)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:35:58.847591 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:35:58.998792 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c81c0af6f084b8db165fd51667b3b54'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:58.999573 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c81c0af6f084b8db165fd51667b3b54'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.045753 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c81c0af6f084b8db165fd51667b3b54'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.058911 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c81c0af6f084b8db165fd51667b3b54'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.065927 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c81c0af6f084b8db165fd51667b3b54'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.072122 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1468/2926] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:35:58 2022] DELETE /identity/v3/users/8c81c0af6f084b8db165fd51667b3b54 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:35:59.327099 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:35:59.341876 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf624-88ea-4a13-9fc3-2047f1bc16d1 service placement] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:35:59.400915 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:35:59.401543 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:59.401821 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:59.402092 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] PATH_INFO: `/v3/projects/8042eb99cd7047d4912b2f4b1e50936e` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:59.403233 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8042eb99cd7047d4912b2f4b1e50936e)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:35:59.405127 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:35:59.485850 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8042eb99cd7047d4912b2f4b1e50936e', 'request_id': 'req-a9b42e11-70a2-4792-8ffb-e631e181e428'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.554482 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8042eb99cd7047d4912b2f4b1e50936e', 'request_id': 'req-a9b42e11-70a2-4792-8ffb-e631e181e428'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.560926 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8042eb99cd7047d4912b2f4b1e50936e', 'request_id': 'req-a9b42e11-70a2-4792-8ffb-e631e181e428'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.573022 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] The token cache is being invalidate because project 8042eb99cd7047d4912b2f4b1e50936e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:35:59.573241 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.573877 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.575385 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1459/2927] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:35:59 2022] DELETE /identity/v3/projects/8042eb99cd7047d4912b2f4b1e50936e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:35:59.579378 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:35:59.580815 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:35:59.598293 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31357d-e05c-4d7b-b9ef-ebe839b9c221 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:35:59.618052 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:35:59.657404 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:35:59.658027 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:35:59.658301 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:35:59.658601 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] PATH_INFO: `/v3/users/6164c0b114624a4b99ea533894a72f0b` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:35:59.659718 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6164c0b114624a4b99ea533894a72f0b)` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:35:59.661688 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:35:59.871786 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6164c0b114624a4b99ea533894a72f0b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.872465 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6164c0b114624a4b99ea533894a72f0b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.943402 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6164c0b114624a4b99ea533894a72f0b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.956186 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6164c0b114624a4b99ea533894a72f0b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.964135 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.notifications [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6164c0b114624a4b99ea533894a72f0b'} {{(pid=81770) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:35:59.970015 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1469/2928] 10.222.0.20 () {58 vars in 1224 bytes} [Fri Sep 9 23:35:59 2022] DELETE /identity/v3/users/6164c0b114624a4b99ea533894a72f0b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:36:00.202556 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] Authenticating user token {{(pid=81769) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:36:00.204029 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.fernet_utils [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81769) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:36:00.217559 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b42e11-70a2-4792-8ffb-e631e181e428 admin admin] Validating token access rules against request {{(pid=81769) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:36:00.235968 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:36:00.276810 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e84ee63259eb48d0a32d4ab69da0f378', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7393fe084b62454e9d96e9d57737a145', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81769) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:36:00.277437 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:36:00.277715 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:36:00.277989 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] PATH_INFO: `/v3/projects/a5245d16c18f43ac9a3c08be6e0fcfd8` {{(pid=81769) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:36:00.279135 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5245d16c18f43ac9a3c08be6e0fcfd8)` {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:36:00.281046 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] RBAC: Authorization granted {{(pid=81769) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:36:00.400811 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5245d16c18f43ac9a3c08be6e0fcfd8', 'request_id': 'req-2bad1106-be7c-454c-b435-a8fba57cf2f7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:36:00.457624 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5245d16c18f43ac9a3c08be6e0fcfd8', 'request_id': 'req-2bad1106-be7c-454c-b435-a8fba57cf2f7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:36:00.463557 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5245d16c18f43ac9a3c08be6e0fcfd8', 'request_id': 'req-2bad1106-be7c-454c-b435-a8fba57cf2f7'} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:36:00.475349 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] The token cache is being invalidate because project a5245d16c18f43ac9a3c08be6e0fcfd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81769) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 09 23:36:00.475630 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:36:00.476787 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG keystone.notifications [None req-2bad1106-be7c-454c-b435-a8fba57cf2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81769) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 09 23:36:00.478277 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: [pid: 81769|app: 0|req: 1460/2929] 10.222.0.20 () {58 vars in 1233 bytes} [Fri Sep 9 23:36:00 2022] DELETE /identity/v3/projects/a5245d16c18f43ac9a3c08be6e0fcfd8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 09 23:36:09.966549 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:36:10.472339 dv-cinder-850321-6-iscsi devstack@keystone.service[81769]: DEBUG dbcounter [-] [81769] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81769) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 09 23:39:27.955736 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] Authenticating user token {{(pid=81770) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 09 23:39:27.957198 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:39:28.095405 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e6ba5-f98b-4e26-88b7-bfc88d803226 admin admin] Validating token access rules against request {{(pid=81770) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 09 23:39:28.152780 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd421f-c70c-4d34-8040-c521e901e38f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a418a2d11a234b4586cb57f1c9a4d780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73855506219b40fdb5a097bdc5cbed81', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81770) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 09 23:39:28.153384 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd421f-c70c-4d34-8040-c521e901e38f service placement] REQUEST_METHOD: `GET` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 09 23:39:28.153745 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd421f-c70c-4d34-8040-c521e901e38f service placement] SCRIPT_NAME: `/identity` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 09 23:39:28.154593 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd421f-c70c-4d34-8040-c521e901e38f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81770) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 09 23:39:28.156386 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.fernet_utils [None req-2dcd421f-c70c-4d34-8040-c521e901e38f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81770) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 09 23:39:28.167446 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd421f-c70c-4d34-8040-c521e901e38f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 09 23:39:28.169009 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd421f-c70c-4d34-8040-c521e901e38f service placement] RBAC: Authorization granted {{(pid=81770) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 09 23:39:28.178991 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: [pid: 81770|app: 0|req: 1470/2930] 10.222.0.20 () {60 vars in 1365 bytes} [Fri Sep 9 23:39:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 09 23:39:38.174671 dv-cinder-850321-6-iscsi devstack@keystone.service[81770]: DEBUG dbcounter [-] [81770] Writing DB stats keystone:SELECT=89 {{(pid=81770) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}